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OBiECTIVES 


PROJECT 

In  April  1978.  a  Minion  Anuranct  Confaranoa  wai  spotMorad  jointly  by  tha  Air  Foioa  Spaca  Division,  and 
Industry  to  discun  fundannantal  issuas  that  would  anhanea  spacacraft  minion  suocan.  Tha  1978  Minion 
Assuranca  Confaranca  rasultad  in  tha  implamantation  by  managamant  of  many  of  tha  raeommandations 
ganaratad  in  tha  confaranoa.  Bacausa  of  tha  suocan  in  1978,  tha  Spaca  Community  consisting  of  Air  Forca 
Spaca  Division,  NASA  and  Industry  Associations  (NSIA  and  AIA)  hava  mutuaily  agraad  to  conduct  Minion 
Atsuranca-1980  to  further  axplora  tha  issuas  affecting  both  Govammant  and  Industry  which  inhibit  mission 
succan  on  Spaca  Programs.  Issues  in  program  managamant,  dnign,  procuramant  craftsmanship,  test 
affactivenan,  motivation,  development  prooan  and  in  other  aran  affecting  minion  succan  will  ba  openly 
discussed  and  anaiyzad  in  workshops  with  audience  participation.  Tha  final  objective  will  be  racomman- 
dations  from  tha  workshops  diractad  at  both  Government  aitd  Industry  management  for  implementation. 

GOALS 

•  Identification  of  obstaclas  and  countar-productiva  practices  having  an  effect  on  mission  attainment. 

•  Development  of  cost  affactive  approaches  or  batter  utilization  of  information  and  resources  to 
enhance  mission  success. 

•  Determination  of  new  or  bettor  application  of  existing  techniques  providing  a  forcing  furtetion  to 
mission  risk  minimization. 

CONFERENCE 

The  purpose  of  this  conference  is  to  provide  the  broadest  exposure  attd  most  tima  efficient  maara  of 
exploring  significant  tasks  necessary  to  achieve  mission  assurance.  This  confarerrea  allows  us  to  utilize  tha 
aggregate  technical  arxi  managerial  talents  of  Government  at«d  Industry  to  provida  recommendations  and 
guidelines  in  areas  of  their  major  concern. 

The  conference  objectives  are  to: 

1 .  Stimulate  exchange  of  ideas  between  Irxlustry  arrd  Government  on  management/ 
technical/acquisition  support  aspects  of  mission  assurance. 

2.  Provide  a  forum  through  which  the  total  community  can  impact  ind  address  problem  areas. 

3.  Provida  a  continuance  to  the  1978  SAMSO/Industry  Mission  Assuranca  Conference. 
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Good  afternoon,  ladles  and  gentlemen.  I 
join  those  who  have  preceded  me  in 
welcoming  you  to  our  second  mission 
assurance  conference.  The  goals  of  this 
meeting  reflect  the  sense  of  team  effort 
that  is  the  only  true  key  we  have  found 
to  mission  assurance. 

The  importance  of  this  team  effort  leads 
me  to  suggest  that  the  theme  for  this 
second  conference  might  be  "The  Human 
Equation." 

It  is  the  human  equation,  or  the  human 
factor,  that  is  most  important  to  mission 
assurance  and  success.  The  human  equation 
is  reflected  in  teamwork,  in  craftsman¬ 
ship,  in  pride  and  in  a  job  well  done  - 
and  in  a  sense  of  responsibility  for  one's 
product,  whether  that  product  is  an  RFP 
or  an  integrated  circuit. 

If,  in  the  collective  wisdom  of  the  experts 
who  are  here  for  the  next  five  days,  we 
find  the  key  to  the  vault  which  houses 
the  secret  to  achieving  teamwork  and 
workmanship  -  then  the  funds  invested  in 
this  conference  are,  indeed,  well  spent. 

In  our  history  chapters,  we  have  some 
striking  examples  of  teamwork  and  pride 
of  craftsmanship  that  have  given  us 
mission  assurance.  I'd  like  to  describe 
one  or  two  to  illustrate  that  it  can  be 
done.  And  I'd  also  like  to  give  an 
example  of  how  not  to  do  it. 

In  early  May  of  last  year,  Mr.  James 
Deering,  an  employee  at  Martin-Marietta 
was  inspecting  hydraulic  tubing  on  the 
first  stage  of  a  Titan  III  space  launch 
booster.  He  noticed  a  large  chip  or  flake 
inside  the  tube  approximately  two  inches 
from  the  end.  The  condition  resembled 
chrome  peeling  from  a  car  bumper.  This 
could  become  a  very  serious  problem  if 
the  "chips"  were  to  break  off  and  get 
lodged  in  a  pump  or  reservoir. 

Immediately,  the  Martin-Marietta  engineers 
inspected  the  tube  stock  -  and  they  found 
that  three  out  of  seven  half-inch  tubes 
had  similar  problems.  All  other  contrac¬ 
tors  using  this  tubing  rapidly  became 
involved.  Working  with  Martin-Marietta, 
Aerospace  Corporation,  and  our  Titan 
Program  office,  the  team  ascertained  what 
size  tubing  was  being  affected.  They 
determined  whether  a  potential  for  a 
catastrophic  failure  was  involved,  and 


they  established  a  criterion  for  accep¬ 
tance  of  new  tubing.  More  importantly, 
they  identified  the  causal  factors. 

You  can  imagine  the  difficulty  of 
inspecting  tubing  after  it  has  been 
formed.  A  very  tedious,  time-consuming 
methodology  was  developed  because  we  had 
to  assure  the  quality  and  integrity  of 
the  hardware.  This  methodology  was 
developed  and  implemented  with  the  full 
cooperation  and  collective  wisdom  of 
everyone  involved,  especially  the 
contractor. 

Next,  we  decided  we  needed  to  change 
out  the  tubing  on  the  vehicles  that  were 
already  stacked  at  both  the  east  and 
west  coasts  so  we  would  have  the  best 
chance  of  mission  success.  We  had  not 
done  this  before  -  changing  out  hydraulic 
tubing  on  the  pad  -  and  we  needed  to 
make  sure  we  did  it  right.  Finally, 
we  had  some  launch  windows  we  had  to 
meet.  Everyone  involved  -  the  engineering, 
management,  and  production  people  - 
worked  twelve-hour  shifts  and  devoted 
their  weekends  in  order  to  meet  the 
schedule. . . 

The  culmination  of  this  effort  was  that 
both  launches  met  their  windows  -  and 
both  launches  were  successful.  That  is 
the  kind  of  work  it  takes  to  get  mission 
assurance. 

As  an  aside,  all  tubing  that  Martin- 
Marietta  had  in  stock  met  the  mil -spec. 

But  we  did  put  out  a  crosstell  throughout 
industry  to  protect  against  recurrences 
in  the  future. 

This  is  an  example  of  teamwork  that  gave 
us  mission  assurance  in  the  true  sense 
of  the  term. 

And  it  was  accomplished  without  a  warranty, 
without  an  award  fee,  and  without  negative 
or  positive  incentive  fees.  It  was  accomp¬ 
lished  by  a  successful  human  equation  - 
people  working  together  who  took  pride  in 
(and  responsibility  for)  their  work. 

I  can  say  that,  early  on,  it  was  my 
distinct  privilege  and  pleasure  to  send 
a  personal  note  to  Mr.  John  Deering  at 
Martin-Marietta,  commending  him  for  a 
conscientious  job  well  done.  Let  me 
describe  another  case  -  a  situation  where 
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a  situation  where  things  didn't  go  so 
well . 

On  another  program,  some  propellant 
properties  were  outside  of  specification. 
Studies  were  conducted  to  improve  the 
formulation;  as  a  result,  representatives 
from  the  contractor,  the  subcontractor, 
and  the  government  got  together  and 
agreed  formally  on  the  definition  of  the 
propellant  formulation  and  the  processing 
criteria. 

One  of  the  processing  criteria  involved 
controlling  the  end-of-mix  temperature. 

One  of  the  subcontractors  stated  that 
end-of-mix  temperature  could  be  controlled 
within  three  to  five  degrees  fahrenheit, 
and  the  company  did  not  request  a  waiver 
to  or  change  in  the  specification 
temperature. 

On  the  very  next  batch  of  propellant,  the 
end-of-mix  temperature  exceeded  the  spec 
limit  that  had  been  agreed  to.  This 
out-of-spec  condition  was  processed  as  a 
normal  material  review  board  action  - 
and  the  program  office  was  not  notified 
until  after  the  propellant  had  been  poured 
into  the  motor. 

Although  there  was  no  violation  of 
procedure  as  such,  there  was  a  violation 
of  the  spirit  of  good  working  relationships 
because  the  representatives  who  had 
concurred  in  the  formal  agreement  were 
not  the  technical  people  who  processed 
the  review  board  action. 

Since  that  time,  we've  made  a  strong  effort 
to  weld  together  an  effective  government- 
industry  team  to  prevent  similar  situations 
in  the  future  -  and  it  has  worked  well. 

In  fact,  that  same  government- industry  team 
has  been  actively  involved  in  cooperative 
and  creative  problem-solving.  When  we 
had  a  failure  during  a  volume  simulator 
ignition  test,  we  lost  a  solid  rocket 
motor  case  -  and  we  suddenly  had  a 
scheduling  problem. 

Fabrication  of  another  case  would  have  been 
both  time-consuming  and  costly.  In  order 
to  obtain  another  case  and  preserve  the 
schedule,  the  government,  the  contractor, 
and  the  subcontractor  agreed  to  limit  the 
skirt  load  test  to  155  percent  of  design 
limit  load  on  the  last  motor  case  -  if 
the  stress-strain  curves  during  test 


followed  the  same  trends  as  in  the 
previous  test.  The  specification  called 
for  "Test  to  Failure." 

All  of  the  parties  involved  witnessed  the 
test.  The  conditions  of  the  agreement 
were  met  -  and  the  case  was  preserved 
for  future  testing  with  no  detrimental 
effect  on  the  program. 

These  are  only  three  examples.  There  are 
many  more  -  on  both  sides  of  the  ledger  - 
that  all  of  you  can  recall  from  your 
own  experiences. 

I'd  like  to  share  another  recent  experience 
with  you  that  illustrates  once  again  the 
importance  of  the  human  equation. 

One  of  our  program  offices  recently 
presented  a  mission  readiness  review  for 
a  spacecraft  launch.  The  program  office 
had  kept  book  on  all  component  failures 
during  the  final  test  flow  and  had  tracked 
down  the  cause  of  failure  in  each  case. 

The  statistics  were  interesting.  Most 
of  the  causes  of  failure  had  by  now 
reached  a  certain  noise  level.  But 
workmanship  showed  a  curious  phenomenon. 

On  the  five  previous  spacecraft, 
workmanship  flaws  were  on  the  decrease  - 
as  one  would  expect  -  a  learning  curve, 
if  you  will.  Yet,  on  this  spacecraft, 
they  showed  a  marked  increase. 

I  should  also  mention  at  this  point  that 
these  workmanship  flaws  were  on  components 
that  had  already  passed  acceptance  tests 
and  had  been  installed  on  the  spacecraft. 

We  found  this  phenomenon  most  perplexing. 

We  don't  know  that  we  really  understand 
the  cause  -  most  probably,  the  number  of 
flaws  increased  because  of  a  turnover 
in  people  on  the  assembly  line  at  the 
vendor's  plant. 

But,  whatever  the  cause.  I've  just 
described  what,  in  my  view,  is  the  larger 
challenge  of  this  conference.  How  can  we 
overcome  the  innate  imperfections  of  the 
human  being?  How  can  we  define  the  human 
equation  that  gives  us  teamwork  and  crafts¬ 
manship  and  rejects  mediocrity  -  the  human 
equation  that  refuses  to  work  at  the  lower 
end  of  the  specification? 
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We  know  that  it  can  be  done  -  I've  just 
described  some  cases  which  Illustrate 
that*  *  * 

I  suggest  the  answer  is  not  in  the 
contract  terms,  although  that  can  help. 

I  would  also  suggest  that  reward  is 
more  important  than  punishment.  But 
overriding  all  of  these  factors  is  the 
leadership  in  industry  and  government 
that  breeds  teamwork. 

Leaders  create  an  atmosphere  of  pride 
and  of  excellence;  they  are  the  ones 
who  set  an  example  for  others  to  follow. 
Their  actions  and  words  demonstrate  that 
mediocrity  is  unacceptable  and  that 
teamwork  is  essential;  their  pride  in 
their  jobs  is  contagious. 

In  our  business,  we  cannot  settle  for  less 
than  the  best  -  the  highest  standards 
and  the  greatest  pride  in  workmanship. 

The  cost  of  failure  is  too  high.  We  know 
we  have  to  do  it  right  the  first  time. 

We  know  there  is  no  recall  after  we  light 
the  fuze.  We  know  that  we  can't  afford 
to  do  it  over _ 

These  are  the  more  unusual  characteristics 
of  this  trade  known  as  the  space  business. 

I  am  proud  to  see  so  many  of  you  here  this 
week.  Your  presence  is  indicative  of 
your  interest  in  our  success. 

Our  record  is  a  good  one  -  and  one  that 
we  want  to  make  better.  If  we  can  do  that, 
we  will  be  providing  our  customer  -  your 
families  and  mine  -  a  better  return  on 
the  investment  of  their  tax  dollars. 

But,  more  important,  we'll  be  making  a 
larger  contribution  to  their  freedom. 

With  those  common  motivations,  we  cannot 
help  but  have  a  good  conference  here 
this  week.  Thank  you  for  coming  -  and 
help  us  work  the  human  equation. 
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MISSION  ASSURANCE  CONFERENCE 

J.  W.  Plummer  Speech 
Hyatt  House,  Los  Angeles 
April  28.  1980 

Considering  the  nature  of  this  meeting 
and  the  organizations  represented,  I 
believe  that  it  is  safe  to  assume  that 
most  of  the  people  present  devoted  their 
time  in  college  to  the  fields  of  science 
and  engineering.  However,  I  suspect  that 
in  such  an  education  ’that  each  of  you  was 
required  to  take  at  least  one  course  in 
psychology.  Well,  if  you  did,  somewhere 
in  the  text  of  material  that  you  studied, 
you  encountered  the  story  of  Pavlov's 
dogs. 

In  Pavlov's  classical  experiment  he  took 
a  very  patient  dog  and  installed  him  in 
the  apparatus  shown  in  order  to  record 
data  on  the  dog's  reaction  to  a  stimulus. 

In  this  case  the  stimulus  being  food  and 
the  reaction  being  saliva  which  was  trapped 
and  measured.  The  dog  was  hungry,  the 
dog  would  see  food;  it  would  salivate 
automati jal’y.  Then  the  experimenter 
rang  an  electric  bell  each  time  the  food 
was  present  and,  of  course,  in  a  short 
ti-ne  tne  dog  reacted  to  the  bell  just  as 
he  had  originally  reacted  to  the  food. 

To  summarize:  an  unconditioned  stimulus 
caused  an  unconditioned  response.  Later 
with  the  bell  -  a  conditioned  stimulus 
created  a  conditioned  response. 

At  Lockheed  we  have  been  concerned  about 
the  general  area  of  productivity  and  have 
an  ongoing  program  dealing  with  employee 
motivation.  We  have  retained  a  very  tal¬ 
ented  industrial  psychologist.  Dr.  Ed  Ruda, 
basically  to  assess  our  managerial  capabil¬ 
ity  and  provide  group  counseling  in  the 
area  of  motivation  for  productivity. 

Group  therapy  no  less  -  perhaps  better 
called  group  psychotherapy.  In  his  work 
at  Lockheed,  Ed  Ruda  develops,  in  a  series 
of  sessions,  the  eight  principles  which 
are  a  guide  to  almost  all  of  human  behavior. 
However,  he  emphasizes  two  basic  principles, 
and  he  quotes  them  using  characteristic 
scientific  terminology  as  follows:  "If 
a  person  behaves  and  gets  rewarded  for  that 
behavior,  the  behavior  will  happen  again." 
Transferring  that  to  the  typical  manager- 
subordinate  relationship,  "To  the  degree 
your  subordinates  are  rewarded  for  an 
action,  that  action  will  happen  again." 

But  to  have  continued  "action"  you  must 


provide  repeated  reward,  that  is  reward  - 
action,  reward  -  action. 

The  second  "theorem"  in  two  parts:  "If 
a  response  to  a  situation  is  followed  by 
a  mild  negative  reaction,  the  response 
will  be  less  likely  to  happen  again.  If 
a  response  to  a  situation  is  followed  by 
a  strong  negative  reaction  (punishment), 
you  do  not  create  an  effective  change  in 
behavior." 

I  am  sure  that  most  of  you  people  here 
have  heard  these  "theorems  of  psychology" 
before.  I  suspect  that  most  of  you  believe 
that  as  you  interface  with  other  people, 
subordinates,  family  and  friends,  that  you 
try  to  utilize  these  principles.  However, 
most  of  us,  in  an  objective  and  private 
inventory  of  our  recent  actions,  would 
have  to  admit  to  some  deficiencies. 

Recently  on  one  of  our  local  broadcast 
radio  stations  a  situation  was  described 
which  relates  to  what  I  have  been  talking 
about  here  today.  It  seems  that  the 
high  level  of  health  care  available  today 
has  reduced  the  number  of  people  who 
become  blind.  As  a  consequence,  there  is 
not  as  much  demand  for  Seeing  Eye  dogs. 

Some  innovative  person  got  the  idea  of 
training  these  dogs  to  assist  the  deaf. 

The  dogs  have  to  learn  to  identify  a 
baby's  crying,  a  doorbell,  telephone  or 
smoke  alarm. 

The  American  Humane  Association  is  expert 
in  training  dogs  but  found  this  task  much 
more  difficult  than  that  required  for  the 
typical  Seeing  Eye  dog  and  they  went  back 
to  analyze  the  basic  principles  of  dog 
training.  In  the  process  they  showed  that 
the  most  common  dog  training  technique, 
namely  the  commonly  used  choke  chain  is 
not  an  efficient  training  method. 

This  type  of  training  was  identified  by 
these  experts  as  the  "pull  and  jerk" 
type  management.  It  is  obviously  negative 
reinforcement  and  may  verge  on  punishment. 
The  results  are  slow  training  and  constant 
negative  reinforcement  to  maintain  training 
levels. 

But,  what  has  all  of  this  to  do  with  the 
subject  of  this  symposium,  "Mission 
Assurance?"  Here  we  have  an  agency  of 
our  Federal  Government,  the  United  States 
Air  Force,  dealing  through  one  of  its 
major  commands  for  the  acquisition  of 
equipment  or  services  from  a  corporation. 


The  corporation  in  turn  probably  consists 
of  thousands  of  people  and  a  very  large 
organizational  structure.  Kind  of  imper¬ 
sonal  !  But  of  course  once  the  project 
is  assigned,  it  is  run  in  the  military 
department  by  a  Colonel  or  Lieutenant 
Colonel  assisted  by  a  military  staff  and 
possibly  some  civilian  experts.  And  on 
the  contractor's  side,  it  is  a  program 
manager  assisted  by  a  group  of  engineers 
and  shop  people.  These  government  and 
contractor  people  in  their  day-to-day 
I  interactions  throughout  the  entire  period 
'  of  performance  must  behave  in  accordance 
with  these  principles  to  maximize  motiva¬ 
tion,  improve  productivity,  and,  ensure 
mission  success. 

The  question  is:  Does  the  SPO  in  dealing 
with  his  prime  contractor  use  modern 
motivation  practices  or  does  he  use  "pull 
and  jerk"  management  principles?  How 
about  APPRO  personnel  in  surveying  a 
contractor's  operations  for  purposes  of 
contract  administration?  How  do  we  prime 
contractors  deal  with  our  subcontractors? 
How  do  we  all  treat  our  subordinates?  In 
all  of  the  above  we  could  impove  motivation 
and  achieve  major  steps  in  productivity 
if  we  were  to  go  back  to  the  common  sense 
principles  of  dealing  with  people. 

I  have  had  the  good  fortune  in  my 
professional  years  of  working  in  the 
engineering  department,  as  a  program 
^  manager,  as  a  subcontractor,  as  a  prime 
contractor,  and  even  for  a  short  time  on 
the  other  side  of  the  table,  with  the  Air 
Force.  In  that  process  I  have  observed 
a  great  many  projects,  some  good  and  some 
bad  -  some  that  achieved  their  mission 
objectives  and  some  that  failed.  One 
strong  positive  correlation  throughout 
this  review  of  my  inventory  of  programs 
has  been  that  where  there  was  a  good  solid 
'professional  "teamwork"  established  between 
the  government  agency  and  the  contractor, 
the  program  was  successful. 

Several  months  ago  two  of  our  LMSC 
experienced  and  successful  program  managers 
reflected  on  the  problems  that  we  had 
experienced  with  subcontractors.  How 
should  we  get  our  message  across  to  them 
on  the  types  of  management  systems  we 
expected  from  them  and  the  control  systems 
that  we  wanted  them  to  implement?  Were 
we  being  consistent  in  our  communications 
with  them  as  we  collectively  dealt  with 
their  contracts  manager,  program  manager. 


controls  manager,  and  project  engineers? 

Were  we  listening  to  their  responses  to 
ensure  that  the  communication  was 
effective?  Were  we  motivating  them  by 
rewarding  them  in  some  manner  for  good 
actions  and  were  we  properly  handling  the 
unsatisfactory  situations?  In  short,  were 
we  building  a  team?  These  reflections  and 
the  analysis  thereon  led  to  the  generation 
of  a  management  tutorial  to  be  used  as 
text  of  a  training  program  for  our  own 
people  for  the  proper  control  of  our 
subcontractors. 

In  discussions  with  General  Henry  relative 
to  the  role  I  might  play  in  speaking  for 
industry  at  the  opening  session  of  this 
conference,  he  suggested  that  I  "tell 
the  Air  Force  people  present  what  they 
might  do  for  the  contractors  in  support 
of  the  common  objective  of  mission 
assurance."  With  such  an  invitation  I 
could  not  turn  down  the  opportunity  to 
use  the  forementioned  training  to  suggest 
to  the  Air  Force  senior  officers,  con¬ 
tracting  officers,  and  SPO's  alike,  things 
they  might  do  to  provide  for  the  kind 
of  working  relations  that  will  ensure 
good  program  performance.  These  sugges¬ 
tions  will  be  under  the  banner  of  "Secrets 
of  a  Successful  Customer." 

Under  the  guidelines  of  the  famous  DoDD 
5000  series  of  documents  for  acquisition, 
the  modern  military  customer  has  good 
access  to  information  of  what  is  going 
on  at  his  contractor's  plant.  But  no 
matter  how  ingenious  the  computer  program 
or  how  well  structured  the  organization, 
the  SPO  is,  in  large  part,  dependent 
on  information  being  volunteered  to  him 
by  the  contractor  personnel .  Assume  the 
contractor  program  manager  comes  to  you 
with  information  of  an  increased  overrun 
or  of  a  slipped  schedule,  or  that  the 
very  critical  first  development  article 
has  been  damaged  by  dropping  it  from  a 
forklift  truck.  How  do  you  react? 

The  natural  reaction  is  to  vent  a  few 
of  your  true  feelings,  and  the  fact  that 
the  program  manager  is  made  to  look  totally 
ineffective  in  front  of  his  peers  simply 
drives  home  the  seriousness  of  the 
situation.  But  the  problem  from  this 
unconstrained  reaction  is  that  program 
manager  will  probably  never  again  come 
forward  willingly  with  negative  information. 
He  would  probably  not  lie  or  deliberately 
cover  up  information  which  is  required. 
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but  he  certainly  would  do  his  best  to 
solve  the  problem  before  he  brings  it  to 
you.  This  takes  time  and  sometimes 
results  in  a  far  more  serious  situation. 
Therefore,  the  obvious  -  don't  behead  the 
bearer  of  bad  news! 

The  proper  solution,  of  course,  is  to 
have  such  a  close  working  relationship 
with  the  company  program  manager  and  a 
mutual  sharing  of  responsibility  on  the 
overall  program  that  you  will  know  almost 
as  soon  as  the  program  manager  does  of 
both  the  positive  and  negative  aspects 
of  the  program.  Should  it  be  necessary 
to  lay  on  the  law,  and  this  of  course  is 
sometimes  necessary,  it  should  be  done 
privately  so  as  to  continue  the  relation¬ 
ship  as  team  members.  Remember  the 
psychology  rule  -  be  mildly  aversive  to 
correct  bad  behavior.  Use  punishment 
carefully  -  and  infrequently. 

Many  years  ago  when  I  was  standing  vigil 
over  one  of  our  satellite  projects  in  the 
Air  Force  Satellite  Test  Center  with 
General  John  Martin,  we  talked  about  the 
relative  capability  of  contractors 
supporting  Air  Force  programs.  He  made 
a  statement  which  I  will  never  forget 
as  follows:  "Remeber,  companies  do  not 
have  experience,  people  have  experience." 

It  is  vital  when  you  start  a  new  program 
to  identify  the  key  players  on  the 
contractors  team  and  go  through  the  process 
of  identifying  strengths  and  weaknesses 
of  each  person.  Identify  those  who  are 
inherently  pessimistic  and  those  who 
are  eternally  optimistic  in  order  to 
provide  adjustment  factors  on  information 
given  by  these  individuals. 

You  should  try  to  understand  that  a 
contractor  person,  whether  an  engineer, 
program  manager,  or  vice  president,  has 
a  set  of  pressures  somewhat  different 
than  yours  in  the  military  world.  Such 
understanding  will  help  to  provide  for 
good  communication  and  trusting 
relationships. 

Finally,  considering  all  of  the  above, 
identify  an  individual  or  an  organization 
that  is  best  to  get  your  message  to  the 
total  contractor  organization  President 
to  support  Manager.  Generally,  I  believe 
that  person  is  the  program  manager,  although 
sometimes  it  could  be  the  contracts  man, 
the  marketeer,  or  even  one  of  the  vice 
presidents.  He  is  a  person  who  has  a 


unique  personality,  ability  to  listen, 
and  a  capability  for  communication  within 
his  own  company;  in  other  words,  manage¬ 
ment  listens  when  he  speaks  to  them. 

In  today's  very  competitive  environment, 
we  contractors  study  your  RFP  very  care¬ 
fully  and  have  a  good  grasp  of  the 
organization  you  propose  to  run  the 
program.  I  believe  that  most  contractors 
fairly  well  duplicate  that  organization 
within  their  own  shops  in  order  to  identify 
counterparts  and  encourage  communication. 
But  considering  the  earlier  point  of 
identifying  the  key  players,  both  the 
contractor  and  the  government  must  work 
to  identify  organizational  parallels 
and  establish  close  working  relationships 
for  each.  All  good  managers  delegate 
responsibilities  to  the  lowest  possible 
level.  More  important  is  that  we  encourage 
communications  to  counterpart  organizations 
at  the  lowest  possible  level. 

This  suggestion  has  to  do  with  getting 
around  and  being  visible  to  the  contractor 
personnel  as  an  important  factor  in  moti¬ 
vating  the  people  who  are  doing  the  work. 

We  know  it  works  well  within  our  own  shop. 
For  example,  with  our  own  management,  the 
simple  act  of  walking  around  and  seeing 
people  and  talking  to  them  about  what's 
going  on  in  their  particular  area,  is 
very  effective  for  improving  motivation 
and  morale.  The  impact  that  you  as  a 
customer  have  at  those  levels  is  one  that 
would  probably  surprise  you  -  maybe  even 
scare  you  if  you  knew  how  really  effective 
it  can  be.  One  little  pat  on  the  back 
for  a  fellow  on  the  production  line  can 
work  miracles  in  motivating  that  indivi¬ 
dual  for  several  weeks  or  even  several 
months. 

Positive  reinforcement  is  your  most 
effective  tool.  And  in  the  process  of 
"tramping  the  orchard"  you  can  learn  things 
about  your  program  that  you  cannot  get 
in  any  other  way.  You  are,  in  effect, 
doing  the  same  thing  a  good  program  manager 
does  with  his  people,  and  the  data  so 
obtained  can  be  used  in  your  discussions 
with  the  program  manager  to  solve  mutual 
problems.  You  build  up  friendships  at 
all  levels  in  the  organization  and  these 
individuals  like  to  talk  about  their 
problems  and  their  successes.  Compare  this 
type  of  communication  to  the  formalized 
quarterly  program  review  which  sometimes 
turns  out  to  be  a  recitation  of  data  from 


a  chart  by  a  person  who  doesn't  really 
understand  the  facts. 

You,  as  the  responsible  government  agent 
in  dealing  with  your  contractor,  have  a 
number  of  means  to  convey  your  prime 
objectives.  The  contract  can  state  them 
explicitly,  you  can  write  letters,  you 
can  have  mass  motivation  meetings,  and 
you  can  have  informal  contact.  All  these 
techniques  can  be  used  to  lend  extra 
emphasis  to  your  prime  objectives.  Of 
course  these  objectives  must  be  well 
thought  out  and  represent  your  true  mis¬ 
sion  priorities.  The  current  management 
technique  of  "management  by  objectives" 
often  times  includes  all  that  is  good  and 
avoids  all  that  is  bad.  Proper  objectives 
should  prioritize  the  things  that  are 
most  important  and  identify  other  factors 
that  might  be  sacrificed  to  the  achieve¬ 
ment  of  high  priority  targets. 

In  preparing  for  this  conference  I 
reviewed  the  proceedings  of  last  year’s 
Industry-SAMSO  Conference,  and  noted  that 
from  the  opening  remarks  of  General 
McCormick,  Jack  Townsend,  and  Ed  Rechtin, 
on  through  the  meeting  there  was  a  very 
substantial  discussion  of  the  use  of 
contract  incentives  on  mission  assurance. 

Workshop  B  under  Colonel  Ken  JuVette  was 
one  of  the  best  discussions  of  the 
mechanisms  of  contract  incentives  and 
product  assurance  that  I  have  read. 

Although  I  personally  feel  very  strongly 
about  the  high  value  of  contract  incen¬ 
tives  for  mission  assurance,  I  will  not 
belabor  the  issue  here.  Clearly,  if 
awards  are  involved  they  must  be  specific; 
measured  without  resorting  to  complicated 
formulas  and  be  inherently  motivating. 

There  are  many  cartoons  depicting  the 
program  manager.  My  favorite  is  the 
individual  called  "Program  Manager" 
standing  in  a  lion's  cage  surrounded  by 
three  snarling  cats.  One  with  the  caption, 
"Cost";  another  caption  "Schedule"; 
another  caption,  "Performance."  How  he 
handles  that  trio  of  problems  determines 
whether  or  not  he  will  get  out  of  the 
cage  and  proceed  up  the  management  ladder. 
The  fellow  depicted  here  faces  the  same 
problems.  In  most  companies  the  program 
manager  has  essentially  the  power  of  the 
general  manager  on  all  aspects  pertaining 
to  the  program.  Clearly,  he  is  devoted 
entirely  to  your  interests  and  it  is  vital 


that  you  understand  that  his  career  depends 
on  your  performance.  On  the  other  hand, 
your  career  depends  equally  on  his  perfor¬ 
mance.  It  is  a  full  time  commitment  for 
both. 

In  the  same  vein  you  should  not  use  the 
program  manager  for  things  that  can  be 
done  equally  well  by  special  assistants 
such  as  a  company  contracting  officer, 
security  officer,  or  other  specialty  areas. 
You,  as  the  "boss"  of  the  situation, 
particularly  during  the  early  honeymoon 
days  of  the  SPO-contractor  relationship, 
can  insist  that  the  program  be  adequately 
organized  to  handle  all  of  the  specialty 
areas. 

Through  the  efforts  of  a  very  large  number 
of  experts  over  the  years  we  have  generated 
a  number  of  management  systems  which  have 
great  value  to  modern  programs  -  if  used 
properly.  Without  going  into  the  details 
so  obvious  to  most  of  you,  we  must  not 
rely  on  systems  or  computer  programs  to 
do  our  management  job.  Clearly,  the 
computer  is  the  best  tool  available  to 
us  to  keep  track  of  vital  data.  Beyond 
the  obvious  high  cost  of  implementing 
special  computer  systems  for  the  control 
of  the  program,  there  is  the  fact  that  the 
maintenance  and  operation  of  such  systems 
do  take  much  of  the  valuable  time  of  the 
team  that  might  better  be  used  in  solving 
problems.  A  large  quantity  of  reports  and 
data  do  not  guarantee  visibility. 

I  want  to  thank  you  for  your  attention  to 
rny  narrating  these  ten  commandments  of 
a  successful  customer. 

In  my  opinion  the  acquisition  methods 
practiced  by  the  United  States  Air  Force 
are  outstanding.  The  very  large  number  of 
complicated  programs  that  have  been  brought 
on  line  is  a  tribute  to  both  the  military 
management  and  the  contractors  that  have 
formed  the  teams.  I  do  not  believe  that 
the  government  uses  "pull  and  jerk" 
techniques.  I  believe  that  most  of  our 
government  and  contractor  actions  recognize 
positive  reinforcement  -  but  it  doesn't 
hurt  to  go  back  to  fundamentals.  The  best 
sources  of  motivation  for  increased  produc¬ 
tivity  and  for  mission  assurance  are: 
positive  reinforcement,  praise,  and 
reward. . .again  and  again. 
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MISSION  ASSURANCE  CONFERENCE 

Remarks  by  Dr.  Williams 
Los  Angeles,  CA 
April  25,  1980 

I'd  like  to  welcome  all  the  attendees  on 
behalf  of  NASA's  administrator.  Dr.  Robert 
Frosch,  to  the  1980  Mission  Assurance 
Conference.  Dr.  Alan  Lovelace,  our  Deputy 
Administrator,  had  desired  and  intended 
to  be  present  to  give  this  keynote  address 
but  the  press  of  business,  budget 
decisions,  etc.  prevented  his  attendance. 

NASA  believes  strongly  in  the  usefulness 
of  the  conference  as  is  evidenced  by 
out  co-sponsorship  and  by  our  co-chair¬ 
manship  of  many  of  the  panels. 

NASA's  enthusiasm  for  the  project  is  not 
based  on  mere  optimism  but  on  the  very 
real  benefits  which  accrued  to  both 
industry  and  government  and  their  working 
relationship  as  a  result  of  actions  stem¬ 
ming  from  the  conference  of  two  years 
ago.  As  stated  in  the  brochure,  one  of 
the  conference  objectives  is  to  "stimulate 
exchange  of  ideas  between  industry  and 
government  on  management/ technical /acqui¬ 
sition  support  aspects  of  mission 
assurance."  NASA  believes  that  it  is 
essential  to  openly  discuss  key  issues 
in  all  the  disciplines  that  are  impacting 
mission  success  from  the  assurance  stand¬ 
point.  Unlike  other  conferences,  this 
one  requires  you,  as  groups  and  panels  in 
the  workshops,  to  develop  realistic 
recommendations  for  implementation  by 
industry.  Air  Force,  and  NASA.  This  is 
your  opportunity  to  speak  out  and  be 
heard. 

Since  this  is  billed  as  a  keynote  speech 
and  since  details  will  be  capably  and 
thoroughly  discussed  in  the  various 
panels,  I  will  limit  myself  to  the  overall 
objectives  of  this  conference  from  the 
NASA  viewpoint  and  to  relating  tc  you 
some  recent  "Lessons  Learned"  on  the 
Space  Shuttle  and  other  NASA  programs 
as  applied  to  mission  assurance  and 
mission  success. 

In  his  letter  of  invitation  to  the  Aero¬ 
space  leadership  and  to  the  NASA  Center 
Directors,  Dr.  Lovelace  suggested  two 
core  questions  which  I  feel  should  be 
kept  in  mind  as  overall  objectives  during 
all  the  activities  and  panel  meetings  of 


the  conference.  Those  two  questions  are: 

1.  How  can  we  improve  mission 
success  and 

2.  What  can  be  done  to  develop 
improved  and  cost  effective 
methods  for  using  information 
and  resources  to  enhance  mission 
success? 

I'd  like  to  add  a  third  core  question  for 
your  consideration  during  your  workshop 
discussions.  This  question  is: 

Why  can't  we  be  more  effective  in 
applying  "Lessons  Learned"  from  past 
space  programs  for  mission  assurance? 

Keeping  those  basic  questions  in  mind,  let 
me  give  you  a  few  of  my  own  thoughts  which 
might  contribute  to  useful  answers. 

Mission  Assurance  is  not  simply  a  matter 
of  Reliability  and  Quality  Assurance  but 
includes  a  variety  of  other  functions  such 
as: 

(1)  Good  program  management 

(2)  Optimal  design,  testing  and 
development  processess 

(3)  Adequate  screening  of  parts 

(4)  Realistic  parts  programs 

(5)  Realistic  Request  for  Proposal 
(RFP)  and  Statement  of  Work  (SOW) 

(6)  Good  materials  control 

(7)  Effective  contract  incentives 

(8)  Effective  monitoring  of  subcon¬ 
tracts  materials  and  pricing 
problems 

(9)  Continued  motivation  and  training 
of  management  and  operating 
personnel 

(10)  Subcontract  Management 

The  list  of  assurance  functions  can  go  on 
and  on.  The  key  point  I  want  to  make  is 
that  mission  assurance  encompasses  every¬ 
one  involved  in  a  program.  I  am  pleased 
to  see  that  this  conference  is  covering 
the  many  disciplines  from  program  manage¬ 
ment  to  contract  incentives  to  test 


effectiveness.  It  is  essential  that  the 
messages  of  mission  assurance  reach  that 
congregation  not  just  the  choir.  Let  me 
discuss  some  of  these  functions  as 
related  to  mission  assurance. 

We  have  used  all  sorts  of  contract 
incentives  from  Cost-Plus-Fixed-Fee  to 
Cost-Plus-Award-Fee  to  Fixed-Price  with 
In-Orbit  Performance  Award  for  the 
purpose  of  motivating  contractor's 
performance.  However,  we  still  have 
design  problems,  manufacturing  defects, 
poor  workmanship,  vendor  problems, 
materials  problems,  etc.’  We  have  not 
been  able  to  translate  contractural 
incentives  for  contractor's  performance 
to  people's  performance.  I'd  like  to 
see  the  3  workshops  on  contract  incen¬ 
tives,  craftmanship  and  motivation  explore 
the  possiblities  of  relating  contract 
incentive  to  people's  perfomance.  I 
don't  mean  Zero  Defect.  Subcontract 
management  seems  to  be  a  continuous 
source  of  many  of  our  mission  assurance 
problems.  We  have  been  able  to  maintain 
good  visibility  on  what  is  going  on  at 
the  prime  contractors.  But  at  the  sub¬ 
contractor,  vendor  and  supplier  levels 
we  do  not  have  the  same  visibility  for 
mission  assurance.  We  are  relying  on 
certificates  of  conformance,  vendor  audits, 
periodic  source  inspection  to  provide 
to  us  a  warm  feeling  that  things  are  OK. 

But  the  feeling  is  not  too  warm.  Over  the 
years,  we  have  had  problems  in  counterfeit 
parts,  inadequate  processing  of  materials, 
mixed  materials,  deliberate  material 
substitution,  etc.  I'm  not  sure  if  the 
present  practices  of  subcontract  manage¬ 
ment  are  adequate.  I  see  from  the  program 
brochure  that  the  conference  has  a  work¬ 
shop  called  "Subcontract  Managment- 
Looking  Up  and  Looking  Down."  I  hope  this 
workshop  will  address  the  improvements  we 
need  in  sub-contract  management  so  that 
we  have  the  visibility  for  mission 
assurance. 

I  believe  that  it  is  essential  for  us  to 
explore  new  and  better  ways  for  mission 
assurance.  However,  we  have  not  taken 
the  "Lessons  Learned"  from  previous  space 
programs  and  applied  what  we  learned  in 
other  areas  and  other  programs. 

Let  me  give  you  some  recent  examples  of 
problems  and  hopefully  some  "Lessons 
Learned"  from  these  problems  for  your 
consideration  during  your  workshops. 


The  Reynolds  Aluminun  Plates  with  Soft 
Spots.  I'm  sure  most  of  you  have  heard 
of  this  problem.  The  impact  of  defective 
aluminum  plates  was  worldwide  affecting 
commercial  and  military  aircraft  and  the 
space  program.  A  major  investigation 
resulted  in  screening  of  aluminum  plates, 
revision  of  inadequate  military  specifica¬ 
tions,  increased  government  and  industry 
inspection,  implementation  of  process 
control  at  the  mill,  etc.  Also,  we  found 
that  conductivity  tests  and  limits  were 
not  adequately  defined.  This  particular 
problem  is  being  taken  care  of  but  what 
are  we  doing  about  the  "Lessons  Learned"? 
We  should  re-examine  the  adequacy  of  other 
material  specifications  and  NDE  methods. 

We  should  consider  ways  of  verifying  to 
accuracy  of  certificate  of  material 
conformance.  We  need  to  apply  the 
"Lessons  Learned"  from  this  problem  to 
prevent  major  problems  in  other  materials 
areas. 

INCO  718  Weld  Wire  Problem.  Some  of  you 
may  not  be  aware  of  this  problem  but 
NASA  lost  a  Space  Shuttle  main  engine 
because  of  mixed  weld  wire  material  at 
supplier.  And  this  problem  is  not 
restricted  to  a  single  supplier.  The 
loss  to  NASA  is  not  just  a  monetary  loss 
in  the  millions  of  dollars,  but  also  in 
the  loss  of  valuable  time  in  the  develop¬ 
ment  of  the  Shuttle.  Although  the  root 
cause  of  the  problem  was  traced  to  a 
lack  of  quality  control  at  the  vendor, 
the  mixing  of  weld  wire  was  known  to 
several  aerospace  companies,  but  not 
reported  via  the  GIDEP  Alert  System.  The 
"Lessons  Learned"  from  this  example  is 
that  aerospace  companies  should  have 
alerted  other  companies  of  the  mixing  of 
weld  wire  materials.  Prime  contractors' 
surveys  of  the  wire  manufacturers  should 
have  uncovered  inadequate  process  and 
quality  control.  We  should  maintain 
visibility  at  third  and  fouth  tier  vend¬ 
ors  and  suppliers  for  mission  assurance. 
This  is  not  a  new  problem.  It  is  my 
understanding  that  mixed  weld  wires  have 
occurred  in  the  past  but  we  have  not 
applied  the  "Lessons  Learned"  from  the 
past. 

Space  Shuttle  Thermal  Protection  or  the 
Shuttle  Tile  Problems.  In  this  partic- 
ular  case,  I 'm  sure  all  of  you  in  the 
aerospace  community  are  aware  of  our 
problems  associated  with  the  Thermal 
Protection  System  (TPS).  The  tile  and 
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other  Shuttle  problems  are  keeping  Dr. 
Frosch  and  Dr.  Lovelace  awake  at  nights. 
From  a  thermal  protection  standpoint,  the 
tiles  are  superior  to  the  materials  used 
in  the  past.  However,  we  are  having 
problems  in  process  control,  in  material 
control,  in  inspectabil ity,  etc.  In 
designing  the  TPS  we  did  not  consider  the 
problems  in  installing  the  thousands  of 
tiles,  in  making  sure  the  tiles  are 
glued  properly,  in  the  handling,  process, 
etc.  The  "Lesson  Learned"  from  this 
case  is  that  in  the  development  process 
from  the  paper  to  the  product,  all  the 
assurance  functions  must  be  considered 
in  order  to  prevent  inherent  problems. 

Cracked  B-nuts  Sleeves.  Recently  during 
final  tests  at  the  Kennedy  Space  Center 
on  a  launch  vehicle  we  found  some  cracked 
B-nuts  sleeves.  This  resulted  in  a 
launch  delay,  which  is  very  costly. 
Investigation  of  the  cracked  sleeve  prob¬ 
lem  revealed  that  the  material  was 
sensitaized.  Further  investigation  found 
wrong  and  sensitized  materials  at  more 
than  one  vendor.  Like  the  mixed  weld 
wire  problem,  cracked  sleeves  have  been 
with  us  for  a  long  time.  We  have  taken 
appropriate  corrective  action  each  time 
the  problem  comes  up  but  we  have  not 
applied  the  "Lessons  Learned"  to  prevent 
the  problem  from  happening  again.  The 
examples  I  have  cited  are  real  problems 
and  have  occurred  previously.  Each  time 
it  happens,  we  put  a  Band-Aid  on,  but 
we  have  not  reached  the  root  causes  to 
prevent  the  reccurance  of  the  problem. 
These  are  not  just  mission  assurance 
problems  but  mission  success  problems. 

The  survival  of  NASA  is  dependent  on  the 
success  of  developing  a  Space  Transpor¬ 
tation  System.  This  requires  mission 
assurance  functions  from  top  program 
managers  down  to  the  floor  workers  at 
the  lowest  vendors  covering  all 
disciplines. 

With  the  framework  of  the  three  basic 
questions  posed  earlier  I  hope  all  the 
workshops  consider  the  "Lessons  Learned" 
from  past  and  recent  programs  in: 

(A)  Realistic  recommendations  for 
implementation  by  government 
and  industry. 


(C)  The  exposure  of  middle  manage¬ 
ment  in  all  disciplines  in  both 
government  and  industry  to 
mission  assurance  problems. 

I  am  looking  forward  to  Friday  to  hear 
your  recommendations  on  mission  assurance. 


(B)  Better  communications  and 
improved  government/ industry 
working  relationships;  and 
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Dr.  Rechtin 

The  central  mission  assurance  problem  of 
Space  Division  and  Aerospace,  as  one  of 
its  principal  advisors,  is  that  about  once 
a  month  we  have  to  certify  readiness  for 
launch.  The  certification  risk  typically 
averages  $200  million  per  crack. 

Since  the  last  Mission  Assurance  Confer¬ 
ence  two  years  ago,  we  have  had  good  per¬ 
formance  on  orbit;  there  have  been  no 
launch  failures.  The  Titans,  Atlases, 

Thors  and  Centaurs  have  all  gone  well. 

It's  a  remarkable  record  -  an  unsung  story, 
really.  Those  vehicles  are  now  approach¬ 
ing  30  years  of  active  life  as  launch 
vehicles  -  something  which  I  think  we  can 
look  forward  to  for  the  shuttle  as  well. 

The  spacecraft  record  also  over  the  past 
two  years  has  been  quite  remarkable.  I 
think  by  anyone's  standards  we  can  give 
4  stars  to  a  number  of  programs.  The  ones 
which  I  am  particularly  impressed  with  are 
the  TRW  DSCS  II's,  the  NATO  Ill's  of  Ford 
Communications  and  Aerospace,  TRW's 
FltSatCom  and  Rockwell's  GPS  R&D  phase. 
Every  one  of  these  are  quite  remarkable. 
Looking  retrospectively,  defense  communi¬ 
cations  satellites  used  to  be  in  some 
disrepute  but  over  the  last  some  years 
on-orbit  performance  by  those  spacecraft 
is  virtually  flawless.  FltSatCom  is  one 
of  the  most  astounding  successes  to  come 
down  the  pike  in  a  long  time.  There  are 
other  programs  that  have  done  quite  well  - 
the  STP,  the  DMSP  and  others.  There  have 
been  almost  no  spacecraft  that  have  not  at 
least  reached  mission  design  life  and  most 
of  them  have  done  better  than  that. 

However,  that  was  the  good  news.  There 
have  been  a  number  of  fairly  serious 
anomalies,  some  of  them  on  orbit.  Most 
anomalies,  as  has  been  earlier  shown  by 
NASA,  occur  on  the  ground  and  one  natu¬ 
rally  attempts  to  clear  these.  A  typical 
average  number  of  anomalies  prelaunch  is 
between  50  and  70. 

There  appears  to  be  some  breakdown  in  the 
procedures  and  discipline  in  the  engineer¬ 
ing  world,  a  great  deal  of  it  at  the 
subcontractor  level.  The  slide  shows  two 
typical  examples.  Somebody  someplace 
didn't  pay  enough  attention.  Fortunately, 


neither  of  these  two  examples  happened  to 
result  in  disaster.  The  other  side  of 
that  coin  is  the  story  that  General  Henry 
told  about  a  conscientious  worker  finding 
a  chip  in  the  tubing  in  time. 

The  TWT  problem  is  a  terrible  alligator. 
It's  been  with  us  for  years.  I  wish  I 
could  say  it  was  going  away.  It  hasn't 
gone  away  yet.  It's  a  difficult  problem 
because  we  are  asking  travelling  wave 
tubes  to  do  things  which  they  normally 
don't  do  in  the  rest  of  their  industry. 

We  are  asking  for  a  very  high  power  and 
long  life  combined,  a  very  tough  set  of 
things  to  do.  We  still  don't  have  com¬ 
petitive  production  of  space  quality  TWT's. 
A  typical  statement  in  the  Readiness 
Reviews  is  that  the  TWT's  are  marginally 
acceptable  but  they  are  the  best  available. 
That's,  if  anything,  a  left-handed 
compl iment. 

We're  not  out  of  the  ball  bearing  problem 
either,  and  are  likely  to  continue  that  as 
long  as  we  have  spinners  or  spinning  plat¬ 
forms  or  rotating  joints.  Most  of  the 
problem  seems  to  be  understanding  the 
usage.  The  analytic  capability  we've 
applied  to  the  ball  bearing  problem  seems 
to  have  been  a  major  problem,  rather  than 
the  materials  as  such.  We  just  haven't 
really  worked  out  what  the  loads  are  well 
enough. 

Contamination  is  a  problem  which  continues 
to  plague  us,  It  always  will  because  it 
involves  contamination  of  some  parts  of 
the  spacecraft  by  other  parts  or  by  launch 
vehicles.  You  can  see  that  there  is  a  new 
problem  which  we  haven't  had  to  worry  much 
about  before,  the  accelerated  outgassing 
of  polymeric  systems  due  to  laser  attack. 
There  is  another  one  at  the  bottom  which 
hit  us  as  a  surprise,  the  initially  harm¬ 
less  transparent  deposits  that  are  around 
that  can  become  opaque  and  harmful  after 
long-term  exposure  to  ultraviolet  radia¬ 
tion.  The  basic  chemistry  changes  in 
those  particular  deposits.  So  what  we 
used  to  think  of  as  an  almost  "safe 
contaminant"  are,  after  some  years  in 
orbit,  proving  to  be  just  as  bad  and  some¬ 
times  worse  than  others.  The  shuttle  is 
going  to  give  us  a  new  set  of  contamination 
problems.  The  shuttle  bay  is  quite 
different  from  being  inside  a  protecting 
shroud  which  has  been  carefully  environ¬ 
mentally  controlled  the  whole  time.  The 
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shuttle  bay  Is  not  that  clean  and  can 
present  serious  problems. 

One  of  the  more  difficult  of  the  future 
challenges  is  the  LSI  and  VLSI  compli¬ 
cation,  We  haven't  paid  too  much  atten¬ 
tion  to  this  yet  but  I  think  it's  catching 
up  with  us  fast.  Unfortunately,  the 
general  thrust  of  the  VLSI  buisness  is  not 
the  same  as  that  needed  for  space  appli¬ 
cation.  The  potential  of  VLSI  is  great 
but  our  space  program  requirements  are 
very  seriously  divergent  from  those  of 
other  users  particularly  in  radiation 
hardness  and  performance  verification.  We 
have  previously  relied  on  quality  control 
techniques,  precap  visual  inspection,  front 
end  electrical  characterization  and  failure 
analysis.  But  these  cannot  as  effectively 
be  applied  to  LSI;  in  some  cases  they  can't 
be  applied  at  all.  New  techniques  have  to 
be  developed.  Custom  designed  LSI  and 
VLSI  could  be  very  attractive  but  design 
verification  is  then  a  major  problem. 

Indeed  for  some  VLSI  units  it's  been  shown 
mathematically  that  complete  certification 
is  impossible.  Obviously,  it's  going  to 
be  very  difficult  for  somebody  who  has  to 
certify  readiness  for  launch  to  have 
components  in  the  system  that  he  knows 
cannot  be  tested  in  all  the  modes  that  the 
components  might  see.  Unfortunately  the 
industry  is  going  to  be  at  a  multibillion 
dollar  level  in  the  mid-80's  and  will  be 
very  resistant  to  perturbations  by  special 
low-volume  customers  -  that's  us!  If  you 
ask  people  in  that  industry  what  they 
think,  they  say  well,  get  some  design  and 
manufacture  it  like  mad  and  hope  for  the 
learning  curve  to  sort  out  the  problems  - 
but  what  if  you  need  quantities  of  only 
ten  to  hundreds?  Obviously  a  much  more 
concerted  effort  is  going  to  be  needed  on 
the  LSI,  VLSI  problem.  And  for  all  those 
happy  engineers  that  say  the  VLSI  is  going 
to  do  wonders  for  the  space  business  - 
yes,  it  may  do  wonders,  but  it's  going  to 
be  one  hell  of  an  alligator  as  well. 

I  can't  just  leave  the  alligators  without 
bringing  up  software.  I  put  up  only  part 
of  the  problem  here,  but  those  of  you  that 
have  had  to  stand  around  and  wait  in  your 
program  because  software  was  the  long  pole 
in  the  tent  know  what  I  mean.  That  part 
of  our  business  is  not  under  too  good 
control;  we  don't  understand  the  manage¬ 
ment  of  very  large  software  systems.  We 
do  know  that  we  are  going  to  be  putting 


more  and  more  things  on  spacecraft.  We 
do  know  that  there  is  a  lot  of  advantage 
to  on-board  computers.  We  are  all  talking 
about  automated  spacecraft;  but,  go  talk 
to  JPL  sometime  on  what  happens  when  this 
fully  automated  spacecraft  refuses  to 
listen  to  what  you're  telling  it  because 
it's  all  automated  and  knows  what  it  ought 
to  do.  Initiation  of  applications  software 
development  before  you  have  the  basic 
tools  built  has  been  a  classic  problem. 

We  talk  a  lot  about  the  distributed  systems 
on  board  spacecraft  and  what  the  subsystems 
might  do  that  hasn’t  been  done  yet.  But 
if  we  commit  to  these  systems  and  don't 
have  the  basic  data  communications  and 
subsystems  architectures  established,  we 
can  be  in  very  severe  trouble. 

We  have  a  couple  of  areas  that  can  pose 
potentially  very  serious  problems.  We’re 
starting  now  into  an  era  where  critical 
raw  materials  have  to  be  imported; 
international  turbulence  is  not  helping 
us  there!  And  as  General  Henry  points 
out  in  almost  every  speech  that  he  gives, 
we  have  a  very  narrow  industrial  base  for 
a  space  quality  hardware.  Typical  scarcity 
lists  include  TWT's,  rubidium  clocks, 
electronic  piece  parts,  solar  cells, 
batteries,  adhesives  and  coatings.  Every 
one  of  these  have  caused  us  trouble. 

Every  one  of  these  are  produced  by  a  very 
limited  number  of  companies.  We  find  our¬ 
selves  in  potentially  serious  trouble 
because  the  industries  that  are  involved 
very  often  are  doing  it,  if  not  as  a  favor 
to  the  space  business,  solely  out  of 
patriotism  and  loyalty  to  national 
security.  That  works  fine  until  a  company 
starts  to  lose  a  lot  of  money.  Then 
management  starts  to  say,  "Well,  there's 
a  limit  to  how  far  we  can  go  in  this 
business. 

So,  the  good  news  is  that  we  have  been 
doing  okay  so  far;  we've  got  problems  that 
keep  staring  us  in  the  face  that  we  still 
haven't  fixed  yet  and  it  takes  gutsy 
decisions  by  program  managers  just  to 
decide  to  live  with  the  risks.  We  have 
new  risks  that  are  coming  up.  The  future 
is  going  to  be  tougher  than  the  past. 


Two-Year  Overview 


•  IASI  MISSION  ASSLIUANOI  roNllDINCI  CONVlNfU  ?5  APKIl  1978 


miRINI.  III!  INIIRVtNING  liA  DAYS  SI'AI  I  DIVISION  PKOOHAMS 
IIAVF  Ivi'IKIINCrO 

.  NO  lAUM  H  lAIUJIOS 

.  MAfjY  lOMI'lliriY  SlICIISSIUI  SI'ACIORAII 
.  NO  I  Al  ASIHUI'IIIC  ON  ORIill  lAllllRIS.  1^ 

.  MANY  ON  UKKII  urul  L|iui:iiil  UMi  ANOMAIIfS.  SOMI 
SLRIOliS  tNOIII,H  U)  Allli  I  Oil  MISSK'N 


•  MISSION  rinniUMANd  IS  (ANSI  lOk  (:ilAi;!ilD  SAIISIACIION 


•  mil  IK'IOOINI  ANUMAIIIS  CONIINNI  10  111  IK'OIIHIING 


Breakdown  in  Procedures  /  Discipline 
Continues  to  Plague  Top  Contractors 

'  POWER  TRANSISTORS  INSTALLED  BACKWARDS 
.  INSTALLED  PER  DRAWINGS 
.  DRAWINGS  IN  ERROR 


MISSING  MYLAR  INSULATING  SEPARATOR  IN 
ElECTRICAL  DRIVE  OT  THRUSTER  VALVE 

.  NOT  CALLED  OUT  IN  DRAWING 


Traveling  Wave  Tubes  (TWTs) 


•  SPACECRAFT  APPLICATIONS  IMPOSE  STRINGENT 
REQUIREMENTS  ON  TWTs 

.  HIGH  OPERATING  TREQUENCY 

.  HIGH  EFFICIENCY 

,  HIGH  POWER 

.  SEVERE  THERMAL  AND  SHOCK  ENVIRONMENT 
.  LONG  LITE 


•  REPETITIVE  PRODUCTION  OF  SPACE  QUALITY  TWTs 
HAS  NOT  BEEN  ACHIEVED 

.  PROBLEMS  IN  MATERIAL  AND  PROCESS 
CONTROLS  DURING  FABRICATION 

.  DEFICIENCIES  IN  TESTING  DURING  BUILDUP 
AND  FINAL  ASSEMBLY 


Ball  Bearings 


•  MANY  SO  PROGRAMS  HAVE  EXPERIENCED  BEARING  PROlilEMS 


.  TE.AL  RUBY 


tU  SAl  COM 
DCSC  II  AND  III 


•  INAOEOUATE  DESIGN  ANALYSIS  USUAL  CAUSE  01  PROBLEMS 

.  LOAD  CAPACITY  DATA  IN  COMMTRCIAl  CATALOGS  NOT 
ALWAYS  APPLICAliir  10  SPACE  APPIICAIION 

.  SOPMISTICAIEO,  BUT  AVAIIAHII,  LOMPUIIR  PROGRAMS 
fREOUCNTLY  NOT  USED 

•  SIRUCIURAI  nrxiBllMY  0(I[N  li.NORiO 

•  UI’GRAOING  Of  ANAlYTtCAl  CAPABIinirS  (consKlerirvj 
striicturjl  Hexil)ilily)  RruOilUf) 


Contamination 


■  t  (TNIAMINAHON  IS  HUtWIM.  A%  |f.MM>«TA\'  mI'.M','  I\  i  AlMT^ATI  |H|’ 

•  IWPAll  OT  ST'A((  STIUUII 

•  CUAMINfNS  OT  (  APl.O  BAY 

•  SHlIlItT  IHfOISTrT^S 

•  OllK.ASSINO  TI^OM  MfiMit 

•  NATlITTAt  OllfT.ASMM.  0»  ST'At*'  'AT'  iMTiYVIPli  .-SMVyiMK  i  IT’.m 
TUIUNM 

•  AT.llin^AMO  iJilIr.AN-.tNr.  lh  i-Oi  >  N'l  l>' K  nyMIV'.  '.t 

•  SOAM  l»llI(.ASSINC.  T*!<l)l)iii  I Wl"  T'.i  T'l  I'l  i  .  1 1>  li  (('<  J’,  tn.-  a’.ii  (IR  'i<f 
(ONtfmi  MllOATfS 

•  -UPAOITT'  I«T‘0'>TTS  iV-VllOAHi.  !ili,T<A|il  '.l'.'iV  •  '  i/i  i  tc  V  AM  T 

•  "HAI^AVtSS  IKANSf’AiOM  IKnixM',  MAt  KliiiVi  ■'•Aij;  }  AM;  mAKMT 

|i)^4T.  ITWM  f  Xi*ciM|T<[  P.iiiv  '.f .  .»s  -is  m.i,',.  .  s  ,  •  ,  ..h  w  oI  .Tpp 


The  LSI  /  VLSI  Complication 


•  IHE  POIfNIIAL  fOR  LSl/VtSi  APPLICATION  10  SPACE  PROGRAMS  (S 
OBVIOUSLY  GREAT 

•  SPACE  PROGRAM  REQUIREMENTS  ARE  SERIOUSLY  DIVERGING  THOM  THOSE  Of 
OTHER  ISI/VLSf  USERS  IN  TERMS  Of  RADIATION  HARDNESS  AND  PERfORMANCE 
VERlMCAflON  (complete  luncttonal  lestinq  ond  char jcteci/jlion) 

•  PREVIOUSLY  RLllEO  UPON  QUALITY  CONTROL  TECHNIQUIS  SUCH  AS  PRECAP 
VI^>UAl  INSPfCIION.  BURN-IN,  ElECIRlCAl  CHARACIER  t/ATlON  AND  lAllURE 
ANALYSIS  CANNOI  AS  filfClIVflY  BE  APIMED  TO  LSI  NEW  lECMNlQUES 
MUSI  BE  DLVllOPID 

•  CUSIOM  I)1SI{,NI0  ISKVISI  Will  BE  VIRY  AIIRACMVE  TO  FUTURE  SYSTEMS. 
HOWEVER.  DfSU-N  VERIMCAIION  WILL  POSE  A  MAJOR  PROBLEM 

•  IME  ISKVISI  INOliSIRt  WILL  BE  OPERATING  AT  A  MUITIBIILION  OOllAR 

ifvfi  IN  The  Min  iq«0'%  and  win  BE  very  resistant  to  perturbations 

I  ROM  SPi  I  IM  lOW  VOlUMl  CUSTOMERS 

•  OBVIUHSiv.  A  MlHM  MORE  (ONCERIfO  EEFORT  ON  THE  PARI  01  THE 
uOVLRNMl  NT  AEROSPALE  '  INDUSTRY  COMPLEX  Will  RE  REQUIRED  TO 
tiLfdlVllY  INCORPORAIE  ISKVISI  INTO  OUR  SYSTEMS 


Computers  /  Software  (s/w) 


•  VIRIur  OF  ON-BOARO  PROGRAMMABLE  COMPUTER  WITH  s/w  UPLOAD 
CAPABILITY  CONVINCINGLY  DEMONSTRATED  IDMSP) 

•  PERMITS  ADAPTATION  OE  SPACECRAFT  SYSTEMS  TO  UNFORESEEN 
CONDITIONS 

•  EFFECTIVE  EXPLOITATION  REQUIRES 

.  TROLIBlE-rREE  SWIICIIINC  BETWEEN  REDUNDANT  COMPUTERS 

.  ADEQUATE  GROUND  FACILITIES  FOR  RAPID  CHECK-OUI  OF 
5/w  TO  BE  UPLOADED 

•  INIIIAIION  OF  APPLICATIONS  SOFTWARE  DEVELOPMENT  PRIOR  TO 
AVAIIABILIIY  Of  MATURE  s/w  TOOLS  STIll  A  PROBLEM  IldCk  0(  mature 
Jovidl  compiler  for  DSP  Small  Processmy  System  (SPSt ) 

•  DATA  COMMUNICAflONS  BtIWEEN  PROCESSORS  IS  STILL  TROUBtfSOME 
(DSP  ■  SPSI 

•  PARKINSON'S  LAW  FOfT  COMPUTERS  ("fecjuirements"  will  jlwdys  cjrow 
to  Sdiufjie  speed  and  memory)  is  SULl.  VALID 

ilUS  computer,  wiTU  4  x  speed  and  4  x  memory  of  Titan  til  computer. 

IS  ■'lull") 


Potential  Problem  Areas 

•  CONTINUING  AVAILABILITY  OF  IMPORTED 
CRITICAL  RAW  MATERIALS 

•  NARROW  INDUSTRIAL  BASE  FOR  SPACE 
QUALITY  HARDWARE 

•  TWTs 

•  Rb  CLOCKS 

•  ELECTRONIC  PIECE  PARTS 

•  SOLAR  CELLS 

•  BAHERIES 

•  ADHESIVES,  COATINGS,  etc 


PAST  AND  FUTURE  CHALLENGES 
Gerald  E.  Lutz 

Director  of  Quality  Assurance 
Fairchild  Space  and  Electronics  Company 
(Vice-Chairman,  NSIA  QRAC) 

Two  years  ago,  almost  to  the  day,  we  con¬ 
cluded  the  first  Mission  Assurance 
Conference.  It  was  an  initial  attempt  to 
cross  discipline  lines  with  a  single 
unique  interest  and  objective.  The  key- 
point  was  that  both  Industry  and  Govern¬ 
mental  agencies  have  a  common  base  to 
perform  a  dialogue  for  a  mutual  goal  - 
MISSION  ASSURANCE. 

Major  General  Howard  E.  McCormick,  past 
Vice  Commander  of  the  Space  Division  in 
his  1978  Welcome  Address  reflected,  "This 
multi-discipline  approach  -  taking  a 
horizontal  cut  at  the  entire  range  of 
problems  that  impinge  on  mission  success  - 
marks  this  as  a  milestone  conference."  He 
further  stated  "Ho  one  realizes  more 
clearly  than  this  assembled  group  that 
attaining  the  quality,  reliability  and 
performance  levels  required  for  our  space 
and  missile  system  is  a  tough  job  -  one 
that  requires  constant  attention  and 
vigilance". 

As  a  result  of  the  recommendations  from 
the  1978  Conference,  Space  Division 
benefits  have  been  cited  as  follows: 

•  Computer  Resources  Department 
established  throughout  AFSC. 

•  Acknowledgement  of  need  to  balance 
performance  with  other  incentives  - 
earlier  incentive  payments. 

•  Acknowledgement  of  award  fee  as  a 
preferred  incentive  method. 

•  An  "Award  Fee  Guidebook"  prepared 
and  released. 

•  SAMSO/ Industry  Committee  is  pre¬ 
paring  a  workmanship  standard  for 
contractual  application. 


•  Aerospace  Corporation  role 
explained  within  contract. 

•  SAMSO  prepared  MIL-S-52779 
Evaluation  Handbook  for  Software 
Quality  Assurance  Program  which 
is  in  final  DOD/Industry  coordi¬ 
nation  cycle. 

•  Software  pamphlets  (QA  and  CADSAT) 
developed. 

•  Buying  Division  QA  Manager 
Training  Program  being  developed 
for  use  throughout  Air  Force 
Systems  Command. 

•  QA  Guidebook  for  Buying  Divisions 
in  final  coordination.  QA 
Guidebook  for  Range  and  Test 
Centers  has  been  released. 

•  Space  Parts  Working  Group  expanded 
for  LSI  and  Hybrids. 

•  SAMSO  encouraging  "Pooled-Procure¬ 
ment"  of  piece  parts. 

e  MIL-STD-1540  under  revision. 

•  Studying  realistic  burn-in 
failure  rates. 

Industry  actions  were  taken  in  the  areas 
of  workmanship  standards  as  they  relate  to 
advancing  technology,  and  subcontractor/ 
contractor  audits/surveys  of  small  business 
ventures,  especially  those  associated  with 
new  technology.  Both  actions  became 
joint  projects  with  NSIA  and  AIA  and  will 
be  addressed  further  within  workshops 
this  week. 

Those  who  attended  the  1978  Conference 
were  asked  to  complete  a  result  oriented 
questionnaire.  Of  the  570  attendees,  70 
percent  responded  to  the  questions  with 
ranked  results  as  follows: 


1.  What  is  highest  priority  issue 
bearing  on  mission  success 
requiring  Space  Division 
attention? 

e  Improve  dialogue  with  Industry 
and  define  priorities  better. 
(If  dialogue  is  improved,  most 
problems  can  be  worked  out. 


•  Lessons  Learned  Program  implemen¬ 
ted. 

t  Improvements  in  Independent  Read- 
idness  Review  procedures. 
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0  Improve  RFP  and  SOW  including 
tailoring  and  application  of 
Specifications  and  Standards. 

0  Educate/ inform  Space  Division 
managers  on  effective 
implementation  of  conference 
ideas 

0  Establish  more  productive 
incentives 

0  Improve  continuity  of  Space 
Division  Program  Management 
personnel 

0  Piece  parts 

2.  What  is  highest  priority  issue 
bearing  on  mission  success 
requiring  Industry  attention? 

0  Improve  motivation 

0  Do  not  buy  in 

0  Improve  design  process  for 
Mission  Assurance 

0  Better  communication  with 
sub-contractors 

0  Better  recognition  that  space 
hardware  requires  tender 
loving  care 

0  Better  workmanship  management 

3.  The  principal  value  of  the 
conference  to  me  has  been: 

0  Free  and  open  exchange  of 
ideas  and  discussion  of 
probl ems 

0  Cross-fertilization  of 
ideas 

0  Learning  what  others  are 
doing  and  better  insight 
into  common  problems 

0  Better  appreciation  of 
Mission  Assurance  problems 
and  interactions 

0  Inspiration 


4.  The  main  message  I  will  take 
back  is: 

0  There  are  many  positive  actions 
I  can  take 

0  Mission  success  is  a  complex 
mutual  problem 

0  Space  Division  is  serious 
(means  business)  about 
improving  Mission  Assurance 

5.  Should  these  sessions  be  con¬ 
tinued?  If  so,  what  format  and 
frequency? 

0  Yes:  100% 

0  Frequency:  Annual 

18  months 
6  mos. 

Format:  Same  format  with 
more  workshop  time 

History  has  reflected  that  results  can 
be  achieved.  Nothing  happens  if  there 
is  no  expectation  -  if  we  are  to  be 
successful,  a  free  and  open  exchange  is 
essential.  Be  aware  that  for  these 
sessions  the  government  is  to  be  treated 
as  a  partner  instead  of  a  customer  -  tell 
each  other  your  concerns.  There  is  an 
outgoing  challenge  to  each  of  you  to 
make  both  good  and  bad  experience  sharing 
happen  so  that  all  parties  benefit. 

You  are  encouraged  to  express  your  thoughts 
throughout  the  sessions.  Attendees  of 
the  1978  Conference  left  having  had  a 
very  refreshing  experience.  We  anticipate 
and  expect  that  you  will  leave  this 
Conference  with  the  same  feelings. 

I  will  now  introduce  the  Chairmen  of  the 
"Past  and  Future  Cahllenges  of  Mission 
Assurance"  panel . 
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PAST  AND  FUTURE  CHALLENGES  OF 
MISSION  ASSURANCE 


Panelist:  Merland  L.  Moseson 

Director  of  Flight  Assurance 
NASA/Goddard  Space  Flight  Center 
Greenbelt,  MD 
April  28,  1980 

Introduction 

Ladies  and  Gentlemen,  first,  I  would  like 
to  express  appreciation  on  behalf  of  NASA 
for  being  invited  to  participate  in  this 
year's  assurance  conference.  In  thinking 
about  what  would  be  most  helpful  in 
getting  this  conference  off  to  a  good 
start  in  the  workshops  planned  for  the 
rest  of  the  week,  I  decided  to  do  three 
things:  First,  to  review  the  Goddard 
flight  record  to  see  where  we  have  had 
problems.  I'll  go  back  to  the  beginning 
in  the  early  1960's.  Second,  on  an 
Agency  basis,  a  study  of  major  problems 
was  conducted  under  the  Chairmanship  of 
our  Chief  Engineer,  Walt  Williams. 

These  problems  were  generally  experienced 
on  the  ground  before  flight  and  the 
results  have  not  previously  had  much 
publicity  except  in  an  internal  briefing 
by  Walt,  to  the  NASA  Center  Directors 
and  to  recap  the  findings  now  should  be 
helpful  for  background  at  this  conference. 
Third,  I  simply  asked  a  number  of  key 
people  from  Goddard's  discipline  organi¬ 
zations  what  they  thought  the  most 
prevalent  problems  were  from  their 
viewpoints.  After  presenting  a  few 
viewgraphs  to  summarize  the  results  of 
this  problem  survey,  I  will  then  propose 
three  challenges  that  seem  to  present 
themselves  from  this  perspective.  I 
hope  that  the  appropriate  workshops  will 
debate  these  challenges  and  refine  and 
promote  them  as  recommendations  from 
this  conference  to  solve  some  of  our 
assurance  problems. 


The  Goddard  Flight  Record 


Brothers  among  the  early  scientific 
satellites.  You  can  also  see  the  earlier 
application  satellites  for  weather  and 
communications.  In  this  group,  we  have 
the  TIROS  and  TOS  series  by  RCA,  the 
Syncom  and  ATS  by  Hughes,  and  NIMBUS  by 
General  Electric  (GE).  There  are  also 
several  international  cooperative 
satellites  shown  including  ISIS,  Alouette 
with  Canada,  FR-1  with  France,  San  Marco 
with  Italy,  and  HEOS  with  the  European 
Space  Research  Organization  (ESRO). 

On  your  right  for  the  decade  of  the 
1970's,  addtional  scientific  satellites 
include  SAS-C  by  APL  and  HCMM  and  SAGE  by 
Boeing.  Also,  the  Earth  observations 
flights  made  by  GE's  Landsat  have  now 
started  in  the  applications  program 
together  with  the  last  Goddard  communica¬ 
tions  satellite,  ATS-6  by  Fairchild, 
and  the  synchronous  orbit  weather  satel¬ 
lite  series  by  Philco  Ford  and  Hughes. 

The  international  programs  continued  in 
the  ANS  program  with  the  Netherlands, 

CAS  with  France,  the  UK-4  and  -5  with 
Great  Britain,  and  Helios  with  Germany. 
Finally,  shwon  on  your  right  is  the  Delta 
Launch  vehicle  by  McDonnell  Douglas  which 
has  flown  151  times  and  has  carried  many 
of  these  and  other  spacecraft  and  now 
represents  a  significant  experience 
base.  These  two  illustrations  depict 
a  record  of  significant  accomplishments 
in  which  we  might  all  take  pride  and  I 
would  expect  virtually  everyone  in  this 
room  has  either  been  personally  involved 
or  works  for  a  company,  agency,  or  govern¬ 
ment  that  has  been  part  of  this  proud 
team.  We  have  worked  on  these  missions 
with  seven  other  countries  on  individual 
projects  and  all  of  the  member  countries 
of  ESRO.  Also,  as  a  major  contributor 
to  this  "Team"  effort,  I  would  like  to 
remind  everyone  of  the  key  role  by  the 
Department  of  Defense  (DOD).  All  of 
these  missions  were  launched  by  Thor, 
Atlas,  Agena,  or  Titan  boosters--al 1  from 
the  DOD  rocket  family. 


The  first  pair  of  slides  illustrates  the 
scope  and  variety  of  U.S.  and  international 
projects  Goddard  has  managed  and  flown 
over  the  past  20  years  and  includes 
examples  of  spacecraft  built  by  virtually 
all  the  major  aerospace  contractors, 
as  well  as  those  built  inhouse  by  Goddard. 
On  your  left  for  the  decade  of  the  1960 's 
you  will  see  the  OAO  by  Grumman,  OGO  by 
TRW,  and  the  origianl  OSO  series  by  Ball 


In  the  next  pair  of  viewgraphs  (II-A  and 
II-B)  is  the  Goddard  flight  record.  On 
your  left  you  see  a  94  percent  successful 
score  on  131  Goddard  spacecraft  flown. 

The  asterisk  means  that  this  is  not  based 
on  the  official  NASA  success  record  which 
is  97  percent  for  these  missions,  because 
in  this  conference  it  seems  more  appro¬ 
priate  to  address  major  inorbit  problems 
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as  failures  even  though  the  mission 
objectives  of  the  spacecraft  were  achieved 
Also,  on  this  screen  the  91  percent 
record  for  151  Delta's  is  given  and  86 
percent  for  2,134  sounding  rockets.  By 
the  ground  rules  used,  we  are  only  scoring 
spacecraft  that  have  been  placed  success¬ 
fully  in  orbit.  In  the  case  of  rockets 
we  have  had  at  least  one  spacecraft 
success  launched  by  a  rocket  that  is 
scored  a  failure  because  it  flew  outside 
the  three  sigma  orbit  planned;  however, 
the  investigators  obtained  satisfactory 
data  in  the  orbit  flown.  '  In  the  case  of 
sounding  rockets,  the  number  means  that 
the  rocket  and  payload  combinations 
both  worked  86  percent  of  the  time. 

On  the  right  screen  our  learning  curve 
appears  to  have  resulted  in  a  success 
record  gain  from  94  to  96  percent  between 
the  first  and  second  decades  of  our 
experience  in  space  and  during  a  trend 
toward  larger  more  complex  spacecraft. 
Similarly  the  Delta  batting  average  has 
grown  from  90  to  94  percent  with  several 
block  change  increases  in  performance 
capability.  Another  difference  shown  is 
that  we  havee  a  100  percent  success  record 
for  the  32  spacecraft  which  have  been 
at  least  integrated  and  tested  inhouse 
versus  93  percent  out  of  house.  From  my 
viewpoint  this  does  not  mean  we  are 
necessarily  better  than  our  contractors 
but  when  we  have  had  an  indepth  involve¬ 
ment  including  the  hands-on  experience 
of  doing  integration  and  testing  in  house 
and  varying  amounts  of  in  house  design, 
we  seem  to  have  been  a  more  effective 
team  "ourselves  and  industry."  The  com¬ 
bined  understanding  achieved  in  this 
mode  appears  to  have  paid  off.  Admit¬ 
tedly  many  of  the  in  house  projects  were 
of  the  smaller  explorer  class  but  included 
several  of  our  larger  and  more  complex 
spacecraft  like  OAO-C,  OGO-III,  OAO-II, 
ISEE,  lUE,  and  SMM.  The  real  objective 
of  in  house  effort,  however,  was  to 
acquire  the  expertise  to  be  effective 
or  "smart"  buyers  of  the  larger  and 
increasing  share  of  our  hardware  which  is 
designed,  built,  and  tested  by  industry. 

I  think  it  is  important,  however,  to  keep 
some  level  of  in  house  activity  in  the 
future  simply  to  maintain  competence. 

The  next  scene  (III-A  and  III-B)  shows  the 
spacecraft  failures  we  have  experienced 
showing  again  the  1960's  period  on  the 


left  and  the  1970 's  on  the  right.  Of  the 
seven  spacecraft  we  have  categorized  as 
major  flight  failures  from  the  engineer¬ 
ing  standpoint,  only  four  were 
categorized  as  mission  failures  by  NASA 
Headquarters,  and  failed  to  meet  their 
mission  objectives.  Even  of  these,  only 
one,  OAO  failed  to  return  any  data.  It 
is  interesting  to  note  that  the  follow-on 
OAO  is  still  in  orbit,  returning  high 
quality  data,  after  about  2,800  days, 
over  7.7  years  in  orbit.  This  is  OAO-C, 
also  called  OAO-3  and  Copernicus  which 
was  launched  August  21,  1972.  Ironically 
we  often  have  real  economic  problems 
with  being  too  successful.  This  is 
because  when  scientific  missions  are 
planned,  they  are  usually  budgeted  for 
a  much  shorter  life  time  in  operations 
and  with  several  of  them  proving  to  have 
considerably  more  longevity  in  orbit, 
their  operations  become  very  hard  to 
support  financially  for  their  full 
lifespan. 

Returning  to  the  viewgraphs  displayed, 
of  the  seven  failures,  three  were  due 
to  early  difficulties  with  three-axis 
stabilization  systems.  The  four  others 
resulted  from  power  problems. 

In  this  flight  experience,  even  when  our 
spacecraft  worked  well,  our  studies 
indicate  that  the  typical  mission  exper¬ 
iences  about  8  or  10  anomalies  during 
the  first  3  years  in  orbit.  Of  these, 
almost  half  occur  in  the  instruments  or 
experiments,  and  the  rest  are  distributed 
among  the  major  subsystems  as  follows; 
C&DH--20  percent,  S&C--14  percent. 

Power — 9percent,  other — 10  percent. 

These  percentages  have  remained  constant 
over  the  years.  The  vast  majorities 
of  these  anomalies,  about  90  percent, 
are  trivial  and  have  no  real  effect  on 
the  mission. 

Taken  collectively,  all  engineering 
disciplines  and  nearly  all  our  contrac¬ 
tors  appear  to  have  been  involved  in  our 
failures  during  ground  and  in  orbit; 
Murphy  travels  everywhere. 

Agencywide  Problem  Survey 

Now  having  shown  you  briefly  the  Goddard 
experience,  I  would  like  to  go  on  to 
step  two  and  take  a  look  from  the  NASA 
as  an  Agency  perspective  based  on  the 


findings  of  a  Failure/Incident  Assessment 
Panel  which  was  convened  in  the  Fall  of 
1978  at  the  request  of  the  Deputy  Admin¬ 
istrator  to  undertake  an  Agencywide 
quick-look  examination  to  determine  if 
failures  and  incidents  were  part  of  a 
pattern. 

For  the  purposes  of  the  panel,  a  failure 
was  defined  as  "the  inability  to  perform 
a  specified,  or  normal  function";  and  an 
incident,  as  "any  unexpected  occurence." 
The  panel,  which  was  chaired  by  the  Chief 
Engineer  of  NASA  and  included  represen¬ 
tatives  from  each  field  center  as  well 
as  NASA  Headquarters,  reviewed  failures 
and  incidents  experienced  within  NASA 
flight  projects  principally  over  the 
previous  12  months  to  identify  underlying 
factors  and  determine  if  correctable 
patterns  existed. 

A  total  of  15  projects  were  examined, 
as  follows: 


The  general  findings  in  the  Fall  of  1978 
were  that  the  occurrence  of  failures  or 
incidents  is  no  greater  than  in  the  past, 
but  that  they  are  occurring  later  in 
the  development  cycle  due  to  reduction  in 
development  testing  and  component  level 
testing.  There  were  also  problems  caused 
by  errors  in  estimating  levels  of  tech¬ 
nical  readiness,  and  lack  of  control  over 
fixed-price  contractors. 

The  field  centers  felt  they  were 
handicapped  by  the  incremental  and  unpre¬ 
dictable  annual  funding  cycle,  hampering 
their  ability  to  plan  total  projects 
intelligently.  Funding  constraints 
result  in  the  deletion  of  analysis  and 
low-level  testing,  resulting  in  paying 
the  price  later  in  high-impact  test 
failures.  Both  QC  and  testing  are 
tempting  areas  to  cut  to  meet  cost  and 
schedule  goals. 

Some  Discipline  Specialist  Opinions 


JPL:  Seasat,  Voyager 

Ames:  Pioneer-Venus,  TRRA,  RSRA, 

QSRA 

GSFC:  Landsat,  lUE,  HCMM,  ISEE 
Johnson:  Orbiter 

MSFC:  Solid  Rocket  Booster,  External 
Tank,  Space  Shuttle  Main 
Engine,  HEAD 

In  'iie  1  ext  pair  of  viewgraphs  (IV-A  and 
IV-B)  is  a  quick  capsule  of  the  results 
from  this  activity.  On  the  left  in  bar 
graph  form  you  can  see  that  nearly  two- 
thirds  of  our  problems  relate  to  design 
with  the  heavier  share  of  design  diffi¬ 
culties  stemming  from  analysis  deficien¬ 
cies  versus  improper  execution  by  making 
errors  or  using  substandard  practices. 

The  other  one-third  of  problems  resulted 
from  test  deficiencies--14  percent  and 
qulity  control  weaknesses--21  percent. 
Further  breakdown  within  these  categories 
is  provided  on  your  right  screen  and 
again  we  see  trouble  scattered  across 
all  disciplines. 

Most  of  these  failures  and  incidents 
occured  during  ground  operations  in 
preparation  for  flight.  Of  a  total  of  70, 
5  were  ascribed  to  failures  or  incidents 
which  are  unavoidable  in  development 
type  work.  The  others  were  categorized 
as  shown  in  the  viewgraphs.  It  was 
concluded  that  the  problems  were  widely 
distributed,  and  had  no  pattern. 


For  the  third  and  last  approach  to 
identifying  weaknesses  in  our  assurance 
program,  I  will  summarize  the  results 
of  soliciting  opinions  from  Goddard's 
specialist  personnel  in  the  disciplines 
of  design,  piece  parts,  materials, 
performance  verification  and  quality 
control  in  the  next  two  viewgraphs  (V-A 
and  V-B).  Under  those  discipline  headings 
are  distilled  quotations  from  persons 
responding. 

Design  Engineers 

—There  is  no  apparent  concentration  or 
pattern  of  design  problems  in  any  speci¬ 
fic  area,  nor  does  the  relative  frequency 
of  design  problems  appear  to  be  changing. 

—True  systems  engineering  is  not  being 
effectively  prosecuted.  System  engineer¬ 
ing  is  pursued  as  a  low-level  effort, 
applied  primarily  in  the  early  stage  of 
a  program.  Systems  are  treated  essen¬ 
tially  as  an  accumulation  of  subsystems. 
The  number  of  "cracks"  increases  as  the 
number  of  subsystem  contractors  increases. 

Parts  Engineers 

—Parts  availability  problems  occur 
because  functional  discipline  either 
does  not  exist  or  is  not  brought  in  on 
timely  or  adequate  basis.  Significant 


problems  occured  in  the  past  5  years  at 
many  of  our-  contractors. 

--Economic  feasibility  problem  for 
suppliers  in  view  of  low  volume  relative 
to  commercial  parts. 

—Rescreening  has  been  found  necessary 
due  to  evidence  of  part  not  being  pro¬ 
perly  processed;  e.g.,  LM108  particle 
probl ems . 

--Two  parts--(a)  Aerospace  contractors 
are  manufadturing  hybrid 
microcircuits  in  many 
instances  without  knowing 
or  utilizing  the  R&QA 
methods  found  necessary  by 
established  microcircuit 
suppl iers. 

--(b)  Increasingly  complex 
VSLI  microcircuits  and  more 
complex  hybrids  call  for  the 
development  of  new  R&QA 
techniques. 

Materials  Specialists 

—Contractors  have  de-emphasized  the  use 
of  materials  specialists,  causing  over¬ 
sights  by  personnel  not  familiar  with 
material  properties. 

—Example:  Reynolds  aluminum  problem. 
Process  was  not  properly  controlled 
or  monitored. 

--Example;  B-nut  sleeve  cracking  problem 
on  Atlas  AC-49  attributed  to  improper 
material  selection,  heat  treatment,  and 
faulty  inspection  that  failed  to  uncover 
the  degraded  material. 

--Example:  The  recent  problem  with 
Shuttle  Orbiter  main  engine  welds  was 
attributed  to  intermixing  two  different 
types  of  welding  rods.  The  distribution, 
receiving,  and  inspection  controls  applied 
apparently  lacked  an  appreciation  of  this 
important  material  control  problem. 

Performance  Verification  Specialists 

—Traditional  Goddard  test  policy 
historically  has  emphasized  testing  at 
the  system  level  under  the  philosophy 
that  it  exercises  all  the  interfaces  and 
provides  a  more  accurate  simulation. 


System  level  testing  is  less  feasible  and 
more  costly  as  payload  size  increases. 

We  must  be  certain  changes  to  this 
policy  now  taking  place  do  not  degrade 
our  mission  reliability  record. 

—The  Navy  now  requires  routine  confidence 
testing  on  many  contracts.  Recent 
problems  with  conductive  particles  and 
poor  workmanship  on  complex  components 
suggest  we  also  should  consider  this 
policy.  But  what  should  the  requirements 
be  and  what  are  the  cost  benefit  trade¬ 
offs  vis-a-vis  our  current  policies? 

--Past  policy  assumed  high  criticality  for 
every  mission.  Shuttle  reflight  capability 
reduces  requirements  for  noncritical  mis¬ 
sions.  Our  problem  is  to  concretely  adjust 
test  requirements  to  reflect  various  levels 
of  mission  criticality  to  optimize  cost 
effectiveness. 

Quality  Control  Engineers 

— Assurance  requirements  are  often 
underestimated  initially  and/or  cut  back 
as  cost  and  schedule  problems  develop. 

—Important  to  apply  QC  engineering  early 
in  the  negotiation  phase  for  long  lead 
and/or  non-standard  items  such  as  hybrids, 
microprocessors,  or  radiation-hardened 
devices. 

Challenges 

Now  I  will  offer  the  three  challenges 
that  I  think  are  suggested  from  the 
problems  we  have  experienced  at  Goddard, 
from  our  Agencywide  survey,  and  from 
assurance  specialist  opinions.  The 
first  one  will  seem  sort  of  motherhood¬ 
like  but  fits  the  general  objective  of 
this  conference  and  this  challenge  is 
presented  specifically  to  place  emphasis 
on  cost  effectiveness  and  is  shown  in 
viewgraph  VI. 

I  believe  on  a  bottom  line  basis  our 
technical  reputation  enjoys  a  better 
rating  than  our  ability  to  control  costs. 
More  importantly,  I  believe  we  as  tax¬ 
payers  have  incurred  avoidable  expense 
and  if  given  a  more  effective  assurance 
program  both  reliability  and  cost  control 
could  improve.  Let's  get  some  good  ideas 
developed  in  the  workshops  of  this 
conference. 
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My  second  challenge  in  the  next  viewgraph 
(VII)  is  not  an  original  idea  with  me 
as  I  think  other  Centers  like  MSFC  and 
Ames  may  be  ahead  of  us  within  NASA  in 
utilizing  the  incentive  approach  and  I 
really  don't  know  how  we  compare  with 
DOD  programs.  Front  end  incentives  for 
assurance  are  very  attractive  to  me 
because  properly  applied,  they  can  make 
it  more  inviting  for  our  contractors 
to  apply  the  needed  discipline  skill 
groups  in  parts,  materials,  and  quality 
control  early  in  the  program.  It  appears 
to  me  they  can  also  help  get  Failure  Mode 
Effects  Criticality  Analysis  (FMECA)  more 
ingrained  into  the  design  activity  and 
promote  more  effective  test  planning  and 
better  assessment  of  the  hardware  adopted 
from  other  programs.  The  other  big 
appeal  about  incentives  is  that  recognizing 
that  we  are  a  big  team  and  with  all  aero¬ 
space  companies  being  more  motivated  on 
front  end  work  emphasis  there  seems  to  be 
an  attractive  potential.  If  in  the  work¬ 
shops  you  will  develop  good  specific 
ideas  along  these  lines,  I  will  help  get 
them  implemented. 

My  last  challenge  is  much  narrower  in  scope 
and  is  basically  the  question  shown  in 
the  next  viewgraph  (VIII).  "Should  we 
institute  confidence  testing  as  a  routine 
aspect  of  the  testing  portion  of  our 
assurance  programs?" 

Confidence  testing,  or  environmental 
screening  refers  to  the  application  of 
environmental  stresses  to  electronic  parts, 
components,  or  subsystem  for  the  purpose 
of  detecting  defects  in  manufacturing  or 
workmanship.  The  tests  are  most  effec¬ 
tive  with  electrical  activation  and 
monitoring  during  test.  The  types, 
levels,  and  durations  of  tests  are 
selected  on  the  basis  of  empirical  or 
other  evidence  of  effectiveness,  without 
regard  for  the  in-service  environment, 
except  that  the  tests  should  not  be  so 
severe  as  to  reduce  the  operational 
life  of  the  tested  item.  The  most  effec¬ 
tive  screens  are  generally  conceded  to 
be  random  vibration  and  thermal  cycling. 

Goddard  policy  has  traditionally  required 
acceptance  testing  at  the  component 
and  system  level  as  well  as  qualification 
testing  for  prototype  or  protoflight 
equipment.  The  levels  and  durations  have 
been  based  on  measured  or  predicted 


flight  environments,  which  are  generally 
more  severe  than  the  levels  usually 
recommended  for  confidence  testing, 
although  usually  shorter  in  duration. 

Only  the  system  level  tests  have  been 
mandatory,  however,  and  practice  on  the 
subsystem  or  component  level  has  varied 
widely  from  project  to  project.  Although 
the  success  of  this  policy  is  attested 
to  by  the  flight  record,  the  increasing 
complexity  of  state-of-the-art  electronics 
and  the  ever  decreasing  size  of  its 
constituent  element  increase  the  proba- 
bilty  of  defects  in  electronics  equipment 
while  at  the  same  time,  decreasing  the 
prospects  of  finding  those  defects  by 
traditional  inspection  and  bench  check 
techniques. 

We  know  it  will  cost  money  to  implement 
confidence  testing  but  will  it  not  save 
more  money  by  avoiding  failures  late  in 
the  build  up? 

As  background  for  considering  this 
question,  we  can  say  historically,  NASA 
instituted  a  highly  successful  screening 
program  for  the  Apollo  mission  which 
consisted  of  a  shaped  random  vibration 
test  and  a  thermal  cycling  test.  The 
results  of  these  tests  were  widely  repor¬ 
ted  in  the  literature  (e.g.,  Simpkinson: 
Testing  to  Insure  Mission  Success, 
"Aeronautics  and  Astronautics,"  March  1970) 

The  Navy  has  now  adopted  a  similar  program 
(Navy  Manufacturing  Screening  Program, 
NAVMAT  P-9492,  Author:  Willis  Willoughby, 
703-692-9058  for  copies).  The  tempera¬ 
ture  cycling  requirements  were  derived 
from  a  Martin-Marietta  report  for  NASA  on 
industry  experience  in  assuring  long-life 
hardware.  The  random  vibration  require¬ 
ments  were  based  on  a  Grumman  study 
summarizing  their  experience  in  the  Apollo 
program. 

A  great  deal  of  research  and  investigation 
is  currently  taking  place  in  the  field 
of  screening.  Most  notably,  the  Institute 
of  Environmental  Sciences  has  established 
a  national  program  on  Environmental 
Screening  of  Electronic  Hardware.  A 
Steering  Committee,  consisting  of  industry 
wide  and  government  representation,  has 
been  appointed  and  has  already  held  at 
least  two  meeting.  The  committee  had 
hoped  to  document  interim  screening  guide¬ 
lines  by  June  of  this  year. 


GODDARD'S  MAJOR  FLIGHT  FAILURES 
INTHE1960*S 


NIMBUS  I 
0/64 


SOLAR  ARRAY  DRIVE 
DESIGN  FAILURE 


3  AXES  STABILIZATION 
FAILURE 


ORBIT  LIFE--1  MONTH 
LIMITED  DATA  RETURN 
MISSION  DECLARED 
SUCCESSFUL  BY  NASA  HQ 

ORBITAL  LIF6-62  MONTHS 
HIGH  EXPERIMENT  DATA  RETURN 


SCANNERS  SENSOR 
RESPONDED  TO  CLOUDS 


ORBITAL  LIFE-28  MONTHS 
HIGH  EXPERIMENT  DATA  RETURN 


POWER  SYSTEM  FAILURE  ORBITAL  LIFE-0 
NO  DATA  RETURN 


FLAT  SPIN  IN  WRONG 
DIRECTION 


ORBITAL  LIFE-117  MONTHS 
SIC  SYSTEMS  OPERATIVE 
AND  HIGH  EXPERIMENT 
DATA  RETURN 


GODDARD  FLIGHT  RECORD 


•  SPACECRAFT  PLACED  IN 
ORBIT 

•  DELTA  LAUNCHES 

•  SOUNDING  ROCKET 
LAUNCHES 


GODDARD’S  MAJOR  FLIGHT  FAILURES 
IN  THE  1970’S 


8/C  a 

LAUNCH 

DATE 

PROBLEM 

NUMBER 

%  SUCCESSFUL 

8A8-B 

FAILURE  OF  LOW 

11/72 

VOLTAGE  POWER 

131 

S4' 

SUPPLY  IN 
EXPERIMENT 

1S1 

91 

AE-D 

LOST  SOLAR  ARRAY 

2134 

86 

10/75 

POWER  -  FAILURE 
IN  POWER  SUPPLY 
ELECTRONICS 

ORBIT  LIFE-61t  MONTHS 
MISSION  DECLARED  SUCCESSFUL 
BY  NASA  HEADQUARTERS 


ORBIT  LIFE-S  MONTHS 
MISSION  DECLARED  SUCCESSFUL 
BY  NASA  HEADQUARTERS 


•V7%  OFFICIAL  SUCCESSES 


%  OF  AVOIDABLE  FAILURES/INCIOENTS 


PROPORTION  OF  FAILURES/INCIDENTS  IN 
EACH  CATEGORY 

(INTERNAL  NASA  STUDY  OF  IS  PROJECTS:  FALL  197^ 


ISSUES 


•  MATERIALS 

—  DESIGNERS  FAIL  TO  USE  MATERIALS  FUNCTIONAL 
SKILLS 

—  SLIP-UPS  IN  PROCESS  CONTROL 

—  UNNECESSARY  USE  OF  STRESS-CORROSION-SENSITIVE 
MATERIALS 

—  FAULTS  IN  INVENTORY  CONTROL 

•  PERFORMANCE  VERIFICATION 

—  HOW  MUCH  LESS  "PAYLOAD  LEVEL"  TESTING  IS 
VIABLE? 

—  SHOULD  WE  DO  MORE  CONFIDENCE  TESTING? 

—  HOW  TO  STRATIFY  REQUIREMENTS  AS  A  FUNCTION 
OF  MISSION  CRITICALITY 

•  QUALITY 

—  PREVENTING  ASSURANCE  PROGRAM  FROM  BEING 
COMPROMISED  BY  RESOURCE  CONSTRAINTS 

—  GETTING  ADEQUATE  ATTENTION  TO  LONG-LEAD 
PROCUREMENT  ITEMS 


CATEGORIES  OF  UNDERLYING  FACTORS 


•  DESIGN  ANALYSIS  DEFICIENCIES _ 29 

-  THERMAL  2 

-  DYNAMIC  9 

-  STATIC  STRUCTURAL  2 

-  TOLERANCE  BUILD-UP  4 

-  SYSTEM  INTEGRATION  ANALYSIS  7 

INTERFACE  COMPATIBILITY 
HAROWARE/SOFTWARE  COMPATIBILITY 
TIMING  ANALYSIS 

-  FAILURE  MODE  AND  EFFECTS  ANALYSIS  3 

-  "OFFTHESHELF-SOITASILITYANALYSIS  2 


•  DESIGN  ERRORS  OR  USE  OF  SUBSTANDARD  (FOR  SPACE) _ 13 


DESIGN  PRACTICES 

-  MATERIALS  OR  DESIGN  MISAPPLICATIONS 

—  MANUFACTURABILITY,  INSPECTABILITY.  OR 
SERVICEABILITY  NEGLECTED 

-  INCOMPLETE  SPECIFICATION  OF  REQUIREMENTS 

•  TEST  DEFICIENCIES _  9 

-  POOR  TEST  DESIGN  4 

—  RELAXED  DISCIPLINE  FOR  DEVELOPMENT  TESTING  4 

—  POOR  PROCEDURES/DISCIPLINE  FOR  PROTOFLIGMT 

TEST  1 

•  QUALITY  CONTROL  DEFICIENCIES _ 14 

•  UNAVOIDABLE  IN  DEVELOPMENT  TYPE  WORK _  S 

70 


ISSUES 


CHALLENGE 

IDENTIFY  AND  PROMOTE  ASSURANCE  TECHNIQUES 
THAT  WILL  IMPROVE  COST  EFFECTIVENESS  AND 
RELIABILITY. 


CHALLENGE 

•HOW  MANY  OF  OUR  ASSURANCE  PROBLEMS  COULD  BE 
ALLEVIATED  BY  APPROPRIATE  FRONT-END  INCENTIVES  IN 
THE  CONTRACTING  SYSTEM? 


•  DESIGN 

-  NO  SPECIFIC  TRENDS  IN  NATURE  OF  DESIGN  PROBLEMS 

-  SYSTEM  ENGINEERING  EFFECTIVENESS  APPEARS  TO  BE 
WEAKENED  AS  A  NUMBER  OF  PARTICIPATING 
ORGANIZATIONS  INCREASE 

•  PARTS 

-  MORE  PARTICIPATION  OF  THE  CONTRACTORS  PARTS 
ENGINEERING  FUNCTIONAL  ORGANIZATION 

-  SHORTAGE  OF  SUPPLY  SOURCES  FOR  HIGH  QUALITY 
STANDARD  PARTS- ESPECIALLY  SEMICONDUCTORS  AND 
MICROCIRCUITS 

-  NEED  MORE  RIGOROUS  QUALITY  ASSURANCE  CONTROL 
FOR  STANDARD  PARTS 

-  ASSURANCE  CONTROL  TECHNIQUES  NOT  KEEPING  UP 
WITH  HARDWARE  TECHNOLOGY 


CHALLENGE 


•  SHOULD  WE  INSTITUTE  CONFIDENCE  TESTING  AS  A  ROUTINE 
PHASE  OF  THE  ASSURANCE  PROGRAM? 


Thomas  A.  Meaker 


structured-industry" .  the  task  of 
evolving  a  specialist  management  system 
Ladies  and  Gentlemen,  it  is  always  a  plea-  is  very  difficult  -  but  absolutely 

sure  to  vis-it  the  UNITED  STATES  OF  AMERICA  necessary. 

and  from  the  experience  of  attending  the 

last  MISSION  ASSURANCE  CONFERENCE  and  In  terms  of  "the  bottom  line  return",  we 

WORKSHOP  it  is  a  particular  pleasure  to  be  are  in  a  minority  section  of  the  industry, 

here  again. 


In  the  tradition  of  this  MISSION  ASSURANCE 
CONFERENCE  I  am  going  to  speak  very  frankly 
and  hope  this  is  the  intention  of  every¬ 
body  for  the  next  four  days. 

I  believe  we  must  continuously  analyse  our 
per7'''rmance  i.e.  our  achieved  results 
against  the  original  requirements  and 
strategies  -  and  the  points  I  am  making 
today  are  based  on  just  such  investiga¬ 
tions  -  particularly  -  concerning  the 
policy  aspects. 

Let  us  in  any  case,  DEFINE  the  MISSION 
EARLY  and  TELL  everyone  involved  WHAT  IT 
IS  ABOUT. 

At  this  time  the  European  Space  Agency  is 
primarily  involved  in  five  main  activities: 

1)  An  application  satellite 
programme. 

2)  The  development  of  space  lab  for 
shuttle  applications. 

3)  A  scientific  satellite  programme- 
part  of  it  in  league  with  NASA. 

4)  The  Ariane  Launch  Vehicle  and  the 
Guyana  Launch  Site. 

5)  An  supporting  research  and 
development  programme. 

For  the  purpose  of  my  contribution  to 
this  conference,  I  would  like  to  state 
that,  I  consider  one  of  the  greatest 
impediments  that  prevents  us  from  obtain¬ 
ing  higher  mission  assurance,  within  the 
decreasing  funding  and  time  available. 
are  inadequate  methods  of  management  and, 
in  particular,  relating  to  the  management 
of  critical  aspects. 

In  a  world  of  increasing  complexity  where 
the  prediction  of  the  future  is  becoming 
increasingly  inaccurate,  and  political 
economics  are  increasingly  dependent  on, 
and  orientated  to.  a  "consumer-product 


In  the  western  world  there  is,  on  average 
more  than  one  major  conference  on  special¬ 
ist  subjects  per  day.  The  only  "realistic, 
conference  that  I  am  aware  of,  where  al 1 
disci  pi ines  can  be  discussed  at  one  time 
is  this  one.  Hence  I  would  encourage  all 
members  of  this  conference  to  keep  the 
following  issue  and  challenge  always  in 
their  peripheral  vision: 

ISSUE:  To  improve  management  with, 

existing  resources,  risk  estimators 
etc.  To  achieve  higher  mission 
assurance. 

CHALLENGE:  To  better  define  the  control 
of  critical  aspects;  be  aware 
of  technique  available  and 
their  limitations. 

It  is  worthwhile  noting  that  our  Data 
Centers  are  saturated  with  techniques 
designed,  and  often  "guaranteed",  to 
prevent  failures  occuring  -  and  yet, 
often  with  agonizing  results,  they  still 
occur.  Not  only  do  failures  still  occur, 
but  they  often  occur  where  it  really 
hurts,  in  ORBIT  ....  or  other  operations! 

For  a  typical  geostationary  telecommuni¬ 
cation  sateTlTteT~thFTlM^3'^<^  resources 
spent  on  problem  solving  prior  to  launch 
is  costing  many  millions  of  dollars;  this 
does  not  include  programme  delays ,  etc. , 
but  refers  only  to  the  identification  and 
processing  of  the  non-conformance  dTta, 
particularly  for  those  events  that  catch 
us  unawares. 

There  is  clearly  something  wrong  with  the 
way  we  handle  our  business. 

My  first  point  is  that  whatever  our 
business,  we  must  have  a  policy,  a  policy 
that  clearly  integrates  all  necessary 
disciplines  at  the  correct  point  in  time, 
and  in  my  opinion,  requires  that  everything 
should  be  related  to  the  hardware  that 
actually  flies. 

My  second  point  is  that  customer  and 
contractor  must  clearly  understand  and 


agree  the  mission  assurance  effects  of 
RFP  clauses  and  bid  commitments  BEFORE 
the  contract  is  signed.  For  example,  the 
meaning,  to  customer  and  contractor,  of 
the  standard  RFP  clause  requiring 
"visibility  and  decision  making 
involvement"  can  be  debated  almost  ad 
infinitum.  It  is  difficult  to  conceive  a 
more  ambiguous  statement  and  yet  that 
statement  alone  can  cost  a  great  deal  of 
"wasted"  mo.iey  by  improper  application 
and  can  in  fact,  reduce  mission  assurance. 

We  must  think  in  terms  of  the  contribution 
these  clauses  make  to  mission  assurance, 
the  positive  influence  they  have  on  hard¬ 
ware  and  software  adequacy  to  meet  the 
mission  requirements.  We  must  try  to 
avoid  contractual  clauses  that  on  the  one 
hand  constitute  self  effacement  and  on  the 
ther  hand,  actually  prevent  the 
implementation  of  efficient  policies 
management  systems. 

It  is  my  belief  that  a  bid  must  clearly 
demonstrate  how  the  management  organization 
intends  to,  practical ly,  function  in  an 
invironment  where  a  simplistic  systematic 
problem  solving  system  cannot,  by  definition 
handle  the  critical  non-systematic  problems 
that  always  occur.  One  aspect  in  which  we 
are  often  delinquent  is  having  system  PA, 
and  Project  Managers  who  are  not  adequatly 
qualified  and  lack  system  engineering  ex¬ 
perience  and  management  training,  and  are 
NOW  AWARE  of  the  value  of  techniques  which 
are  available.  Another  aspect  which  receives, 
in  my  opinion,  insufficient  attention  at  the 
design  stage,  is  hardware  engineering.  "Is 
it  being  put  together  properly?"  How  critical 
is  workmanship  and  does  it  have  a  different 
criticality  in  different  areas? 

My  discussion  so  far  has  mentioned  some  of 
the  problems  and  constraints  we  are  all 
faced  with  -  most  of  them  obvious;  BUT, 
what  is  our  most  critical  and  valuable 
resource  -  a  resource  that  contains  the 
following  terms: 

It  is  ...  polarized 
variable 
indispensable 
unpredictable 
logical 
irrational 


emotional 
unrel iable 
dependable, 

and  contains  extremely  complex  interfaces. 

That  resource,  gentlemen,  is  US.  1 1  i s 
the  resource  we  know  least  about  and  yet 
we  must  also  classify,  integrate,  schedule, 
develop,  motivate  and  be  managed  by  that 
resource  in  order  to  achieve  a  high  mission 
assurance  of  our  product:  the  expert 
whose  views  are  ignored  first  time, 
probably  won't  try  to  contribute  effec¬ 
tively  in  the  second  round;  A  world  of 
PHD'S,  and  managers  will  not  give  us 
mission  assurance.  Personal  experience, 
therefore,  is  also  a  critical  resource. 

So  much  for  talk  and  suggestion.  At  the 
European  Space  Agency,  I  have  attempted 
to  implement  some  of  the  very  good  ideas 
from  the  U.S.  Aerospace  and  elsewhere. 

The  implementation  which  is  still  being 
developed  consists  of  the  production,  in 
the  bid  phase,  of  a  critically  manual ; 
a  blank  page  of  which  is  on  the  screen 
now;  The  manual  constitutes  a  management 
tool ,  by  which  resources,  by  type  can  be 
defined  and  deployed,  according  to 
equipment/mission  critical ity  and  there¬ 
fore  problem  occurance.  It  should  be 
noted  that  all  areas  of  expertise  are 
involved.  A  "design  of  test"  (i.e. 
verification  of  the  test  plans  by  compar¬ 
ison  with  the  FMECA)  is  involved.  Priori ty 
and  phasing  of  inspection  of  hardv/are  is 
determined  as  a  function  of  the  criticality 
of  the  design,  local  technology,  workman¬ 
ship  (solder  joint  in  the  limit)  etc.  to 
mission  success.* 

*Fol lowing  a  number  of  requests,  I  have 
included  a  more  detailed  explanation  of 
the  criticality  management  system  as  an 
appendix  to  this  paper. 

From  my  experience,  management  must  develop 
a  clear  policy  and  criticality  assessment 
tools.  The  tool s  must  be  dynamic,  the 
management  flexible  but  the  appl ication 
of  the  pol icy  quite  rigid.  Scarcity  is 
criticality  and  must  be  considered  in  the 
endeavor  to  improve  mission  assurance.  As 
project  time  elapses,  an  Einsteinian 
E  -  MC2  relationship  seems  to  develop  with 
respect  to  resources  needed  to  solve  late 
problem  occurances  i.e.,  resources  needed 
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to  solve  late  problem  occurances  i.e.; 
resources  needed  astronomically  expand  the 
later  the  problem  occurs  or  is  identified. 
In  many  cases  these  events  can  cause 
corporate  reactions  which  can  not  only 
reduce  mission  assurance  to  an  unacceptabl 
low  level ,  but  can  phase  us  out  of  the 
business.  Let  us  therefore  realize  that 
the  occurance  of  a  particular  problem  at 
different  points  in  a  programme,  can  change 
its  critical ity  and  impact  very 
significantly. 

In  conclusion,  I  consider  that  the  proper 
appl ication  of  product  assurance  standards 
requires  informed,  innovative,  and 
flexible  thinking.  I  even  would  apply 
this  consideration  to  the  qualification  of 
technology  where  I  believe,  we  in  the 
space  industry,  must  look  "wi de-band"  at 
industrial  efforts  in  order  to  use  a 
maximum  amount  of  information  without 
dupl ication. 

I  believe  that  we  have  done  a  good  job, 
when  we  have  achieved  our  mission  assurance 
targets,  then  we  deserve  a  pat  on  the  back. 
At  this  point  in  time,  at  the  commencement 
of  this  very  worthwhile  conference,  which 
is  so  dependent  on  all  our  efforts  to  be 
successful,  let  us  not  pat  ourselves  on  the 
back  until  the  work  is  done. 

I  truly  compliment  and  congratulate  the 
initiators  and  organizers  of  this  event. 
They  say  it  takes  a  genius  to  spot  the 
obvious,  now  that  it  has  been  conceived, 
this  conference  clearly  is  an  obvious 
requirement  to  us  all. 


Thank  you. 


APPENDIX 


CRITICALITY  MONITORING  SYSTEM 


In  order  to  try  and  assess  the  criticality 
of  various  hardware  elements  in  a 
spacecraft  programme  and  to  incorporate 
into  that  criticality  assessment  the 
results  of  the  analyses  and  the  inspection 
activities,  a  monitoring  manual  was  con¬ 
structed  with  a  rating  system  against  the 
various  criteria.  The  criteria  covered 
the  manufacturing  and  hardware  history 
of  the  particular  company  involved,  the 
design  and  in  particular  the  design 
reliability;  The  use  of  parts  materials 
and  processes,  the  non-conformance  history 
of  the  particular  equipment,  parts 
information  in  general,  and  also  CADM 
information.  The  manual  consists  of 
one  page  per  equipment  type  and  covers 
every  equipment  type  on  a  particular 
satellite.  A  page  of  Issue  1  of  the 
manual  is  shown  attached  to  this  paper 
and  it  will  be  noted  that  against  four 
of  the  criteria  a  rating  value  is  given; 
the  rating  constitutes  a  selection  of 
a  number  between  1  and  3  with  the  most 
critical  rating  being  at  the  lowest 
number,  i.e.  the  most  critical  elements 
have  a  rating  of  1.  The  rating  is  applied 
by  the  expert  concerned  after  the  expert 
has  been  instructed  on  the  use  of  the 
manual  and  the  scope  of  the  rating  system 
by  the  Product  Assurance  Manager  concerned. 
It  will  be  seen  that  an  overall  rating 
for  the  equipment,  which  is  arrived  at 
by  arithmetically  averaging  of  the 
individual  ratings,  is  contained  on 
each  page. 

You  should  also  be  noted  that  there  is  an 
area  for  "qualified  remarks"  i.e.  to  cover 
areas  which  are  not  specifically  covered  or 
which  are  not  covered  in  sufficient  detail 
by  the  other  windows  of  the  page.  From  an 
analysis  of  the  individual  rating  assess¬ 
ment,  equipment  specific  monitoring  require¬ 
ments  for  ESA  representati ves  are  written 
down  at  the  bottom  of  each  sheet.  By  this 
method  a  particular  part  of  an  equipment 
can  be  identified  as  being  of  particular 
criticality  and  this  therefore  would  be 
given  a  specific  inspection  point. 

Where  it  is  considered  that  the  design  is 
more  critical  than  the  hardware  then  the 
design  aspact  would  receive  priority 
attention.  The  overall  intent  is  to 


enable  an  engineer  or  manager  to  have 
rapid  visibility  of  the  most  critical 
elements  of  the  spacecraft  and  in  addition 
to  be  informed  of  where  and  why  those 
particular  areas  are  critical. 

Following  on  this  it  is  possible  for 
management  to  identify  particular 
critical  areas  and  to  schedule,  a 
priori,  resources  to  cover  those  needs. 

In  practice  the  most  critical  elements 
within  the  manual  are  brought  together 
and  are  monitored  prior  to  the  monitoring 
of  other  areas. 

It  has  to  be  noticed  that  the  manual  must 
be  maintained  in  an  updated  condition 
and  that  the  priorities  for  particular 
expertise  and  a  particular  monitoring 
application  may  change  as  the  project 
advances. 

If  any  more  detailed  information  is 
required  on  this  system  then  the  author 
would  be  pleased  to  discuss  the  subject 
matter  on  a  case  by  case  basis. 
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GOVERNMENT  ROLES  AND  RESPONSIBILITIES. 

QUANTITY  AND  QUALITY,  OF  GOVERNMENT 
INVOLVEMENT  IN  ASSURING 
CONTRACTUAL/MISSION  PERFORMANCE 

Howard  T.  Wright 
Director  for  Projects 
NASA-Langley  Research  Center 

The  success  of  any  mission  that  involves 
a  complex  assembly  of  hardware  and 
software  is  usually  totally  dependent 
upon  all  of  the  elements  of  the  system  to 
perform  as  desired.  There  is  therefore 
no  one  part  of  a  program  that  can  be 
considered  more  important  than  another 
oecause  each  is  like  a  link  in  a  chain 
and  all  must  be  present  if  the  chain  is 
to  perform.  The  roles  and  responsibil¬ 
ities  of  the  government  in  the  management 
and  acquisition  of  a  system  has  varied 
significantly  in  the  past.  In  some  cases 
the  government  has  taken  on  the  respon¬ 
sibility  for  the  integration  of  the 
efforts  of  many  contractors  and  has  played 
a  significant  role  in  the  total  program 
management.  With  this  approach  the 
government  is  intimately  involved  on  a 
daily  basis  and  must  be  prepared  to 
adequately  staff  a  project  office  with 
qualified  people  in  order  to  provide  the 
necessary  direction  to  the  many  contrac¬ 
tors  involved.  In  this  type  of  program 
management  the  government  is  clearly 
responsible  for  establishing  the  funda¬ 
mental  requirements  for  such  things  as 
subcontract  type,  interface  control, 
configuration  control,  performance 
specifications,  quality  assurance  policy, 
etc.  Recently  there  has  been  a  trend 
toward  less  direct  government  involvement 
by  contracting  with  a  prime  "system 
integration"  contractor  and  requiring 
the  contractor  to  deliver  an  end  product 
to  meet  a  performance  specification  with 
very  little  government  involvement  along 
the  way.  Either  approach  can  be  success¬ 
ful,  however,  there  are  fundamental 
ingredients  to  mission  assurance  that  must 
be  present  for  success  and  the  government 
must  be  reponsible  either  in  a  very 
direct  way  if  the  first  approach  is  used 
or  the  government  must  be  responsible  for 
the  seleccion  of  a  prime  system  contactor 
that  has  the  experience  and  the  necessary 
ingredients  for  success.  In  any  event 
there  must  clearly  be  someone  in  charge 
of  the  program  and  that  someone  has  the 
responsibility  for  the  many  management 


decisions  that  will  affect  the  course 
of  the  program.  The  following  cortiments 
are  based  on  the  author's  experience  on 
programs  in  which  the  government  has 
played  a  significant  role  in  the  manage¬ 
ment  of  the  project.  It  is  believed 
that  the  same  general  comments  would 
apply  to  the  'Systems  Integration" 
contractor  if  the  government  elects  to 
contract  for  that  service. 

1.  Balanced  Program.  -  The  success  of 
any  program  is  dependent  upon  the 
proper  balance  between  the  forces  that 
would  influence  the  technical  performance, 
the  cost  and  the  schedule  accomplishment. 
One  successful  project  manager  always 
referred  to  these  three  items  as  the 
three  legs  on  the  milk  stool,  where 
clearly  the  utility  of  the  stool  is 
dependent  upon  each  of  the  three  legs 

to  the  same  extent.  The  government's 
responsibility  in  establishing  a  contract 
must  clearly  include  a  tho.'ough  knowledge 
and  understanding  of  the  costs  involved 
and  of  the  schedule  performance.  All 
too  often  a  program  is  established  on 
the  basis  of  technical  requirements  and 
arbitrary  or  unrealistic  cost  and 
schedule  requirements  are  applied. 

When  unrealistic  technical  performance 
requirements,  cost  requirements  or 
schedule  requirements  are  established 
by  the  government,  excessive  pressures 
are  created  on  the  subcontractors, 
and  the  quality  of  the  end  product 
is  bound  to  suffer  as  a  result.  The 
government's  responsibility  for  assuring 
a  balanced  program  does  not  stop  at 
the  issuance  of  a  contract,  but  rather 
requires  continual  vigilance  through¬ 
out  the  life  of  the  program  with  many 
changes  and  decisions  being  made  to  be 
sure  that  there  is  always  the  proper 
balance  between  these  three  essential 
elements  of  any  successful  program. 

2.  Selecting  the  People.  -  One  of  the 
first  responsibilities  that  the  govern¬ 
ment  has  in  establishing  a  new  program 
is  to  select  the  people  who  are  to  be 
responsible  for  the  management  of 

the  program.  It  is  important  that  this 
selection  be  done  on  the  basis  of 
experience  and  demonstrated  capability. 

All  too  often  people  are  selected  on 
the  basis  of  availability,  and  thus  a 
program  may  suffer  due  to  lack  of  the 
necessary  experience  and  know-how 
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to  accomplish  the  objectives  of  the 
program.  The  government's  program  for 
merit  promotion  goes  a  long  way  toward 
the  solution  to  this  problem;  however, 
there  is  clearly  a  need  for  greater 
mobility  for  people  within  the  govern¬ 
ment.  Because  of  variations  in  cost  of 
living  between  different  parts  of  the 
country,  it  is  currently  very  difficult 
to  staff  some  government  positions  with 
the  most  qualified  people.  For  those 
cases  where  it  is  not  possible  to  obtain 
the  level  of  experience  or  expertise 
required  to  fulfill  a  job  assignment,  it 
is  the  responsibility  of  the  government 
to  provide  adequate  training  for  the 
individuals  who  are  assigned  to  the 
program. 

3.  Cost  Estimating  and  Management.  - 
An  extremely  important  part  of  the 
government's  responsibility  in  establish¬ 
ing  a  new  program  is  to  determine  a 
realistic  cost  for  the  program.  A  healthy 
contractor  who  is  making  a  profit  is  a 
far  better  and  more  responsive  contractor 
than  one  who  has  overrun  a  fixed  price 
contract  and  is  spending  his  own  money 

to  meet  the  program  objectives.  It  is 
the  government's  responsibility  to  take 
those  steps  necessary  to  insure  that  all 
contractors  have  an  opportunity  to  make 
a  reasonable  profit.  This  responsibility 
is  a  continuing  one,  and  cost  management 
throughout  the  life  of  the  program  is 
important  if  the  government  is  to  be 
successful.  Cost  offset  programs  descope 
options,  performance  relaxation,  etc., 
are  some  of  the  options  available  to  the 
government,  and  all  must  be  under  contin¬ 
uous  review  throughout  the  life  of  the 
program. 

4.  Contracts.  -  The  government  has  a 
responsibility  to  determine  the  type  of 
contract  to  meet  the  specific  needs  of 

a  program.  Contracts  must  be  written  in 
such  a  way  that  contractors  are  required 
to  provide  the  government  with  the 
information  necessary  to  properly  assess 
the  contractor's  performance  throughout 
the  life  of  the  contract.  Different 
types  of  contracts  may  be  appropriate 
for  different  kinds  of  programs,  and  the 
government  is  responsible  for  the  selec¬ 
tion  of  the  type  of  contract  that  best 
satisfies  the  program  needs. 


5.  Communications.  -  Some  programs  in 
the  past,  particularly  large  ones,  have 
suffered  significantly  due  to  a  lack  of 
adequate  communication  between  various 
participating  groups.  The  government  is 
responsible  for  the  establishment  of  a 
communications  system  that  will  provide 
the  opportunity  for  problems  at  all 
levels  of  the  organization  to  be  heard 
at  the  proper  level,  and  when  decisions 
are  made  the  communications  system  must 
communicate  with  all  of  the  parties 
affected  by  the  decision  with  a  minimum 
of  delay.  The  government  is  responsible 
for  the  establishment  of  specific  review 
meetings  at  various  stages  of  the  program. 
This  responsibility  includes  assurance 
that  the  proper  people  are  in  attendance 
at  review  meetings,  that  formal  documen¬ 
tation  is  established  for  all  action 
items,  and  that  decisions  on  all  action 
items  are  made  in  a  timely  fashion  and 
formally  conmunicated  to  all  parties. 

6.  Test  Philosophy.  -  The  government 
is  responsible  for  the  establishment 

of  a  test  philosophy  to  meet  the  require¬ 
ments  of  the  program.  It  is  easy  to 
say  that  the  system  hardware  must  be 
demonstrated  to  perform  as  required  in 
the  environment  in  which  it  is  to  be  used, 
and  that  a  test  program  be  devised  to 
qualify  the  hardware  for  that  environment. 
In  many  cases  it  is  difficult  to  specify 
the  total  environment  for  the  end  use  of 
the  hardware  since  many  elements  of  the 
environment  may  not  be  well  known.  It  is, 
nevertheless,  the  government's  respon¬ 
sibility  to  determine  what  the  environment 
will  be  to  the  best  of  its  ability  and 
establish  a  test  philosophy  that  will 
insure  the  performance  of  the  hardware 
in  the  real  environment. 

7.  Failure  Analysis.  -  One  area  in  which 
many  programs  have  had  difficulty  has 
been  the  establishment  of  an  adequate 
failure  analysis  program  for  the  hardware 
being  developed.  The  government's  respon¬ 
sibility  may  be  to  establish  the  basic 
philosophy  for  failure  analysis,  or  in 
many  cases,  it  may  be  necessary  for  the 
government  to  actually  conduct  failure 
analysis  for  those  contractors  who  may 

be  unable  to  adequately  conduct  their  own 
failure  analysis.  It  is  important  for 
the  government  to  establish  a  baseline 
hardware  configuration  and  carefully 
track  all  failures  from  an  established 
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point  in  time  to  be  sure  that  adequate 
corrective  action  has  been  taken  to  pre¬ 
clude  a  recurrence  of  the  same  failure. 
Although  this  sounds  very  simple  and 
straightforward,  it  is  remarkable  how 
many  programs  have  existed  without  the 
necessary  discipline  in  a  failure  report¬ 
ing  and  analysis  program. 

8.  Providing  Help.  -  The  government  has 
a  responsibility  and  a  unique  capability 
to  provide  "help"  to  contractors  who 
are  working  on  their  program.  Many 
contractors  have  limitations  in  certain 
areas  and  the  government,  because  of  its 
size  and  diversity,  has  a  unique  ability 
to  provide  assistance  in  almost  any 
technical  or  managerial  area. 

9.  Advisory  Group.  -  For  large  and 
complex  programs,  the  government  has  a 
responsibility  to  assemble  and  manage 
advisory  groups  that  have  unique  exper¬ 
tise  in  the  various  elements  of  a  program. 
Very  often  such  advisory  groups  can  be 
small  and  need  to  meet  infrequently  at 
specific  times  in  the  program  as  desig¬ 
nated  by  the  program  manager.  In  other 
cases  it  may  be  necessary  to  establish 
working  groups  of  people  to  concentrate 

on  a  problem  area  for  a  significant  period 
of  time.  In  any  event,  it  is  the 
government's  responsibility  to  see  to 
it  that  the  proper  expertise  is  applied 
to  any  difficulties  that  may  arise  during 
the  course  of  the  program. 

The  above  responsibilities  and  others 
will  be  discussed  in  the  workshop  titled 
"Government  Roles  and  Responsibilities, 
Quantity  and  Quality,  of  Government 
Involvement  in  Assuring  Contractual/ 
Mission  Performance." 
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able  to  afford  one  airplane  per 
year.  Since  many  of  us  are 
engineers ,  we  can  extrapolate  that 
data  fairly  easily. 


JAMES  D.  CLOUD 

ASSISTANT  GROUP  EXECUTIVE,  EDSG 
HUGHES  AIRCRAFT  COMPANY 

Why  is  cost  credibility  so  much 
more  important  today?  One  pri¬ 
mary  reason  is  the  spiraling 
rise  in  the  cost  of  defense 
systems  including  those  costs 
which  are  necessary  to  ensure 
and  assure  mission  success. 

After  all,  part  of  the  cost 
rise  in  aircraft,  as  depicted 
in  Figure  1,  has  been  due  to 
your  requirements  for  assurance 
that  the  weapon  systems  which 
have  an  ever  increasing  cost 
associated  v/ith  it,  can  and 
will  perform  its  intended 
function.  Obviously  such  a 
rise  in  the  cost  of  systems 
must  be  driven  by  increased 
requirements  for  perfoirmance, 
capability,  and  reliability 
thus  resulting  in  fewer 
systems  to  accomplish  the 
intended  functions.  This  is 
obviously  borne  out  by 
Figure  2.  There  are  doubters, 
however,  who  claim  that  it  is 
not  the  increased  performance 
which  is  reducing  the  required 
number  of  aircraft  but  the 
dollars  available  to  purchase 
aircraft . 

This  is  an  alarming  chart  and 
it  is  statistics  such  as  this 
that  I'm  sure  led  General  Slay 
in  his  recent  speech  to  the 
Reliability  and  Maintainability 
Symposium  to  state  that  "If 
things  keep  going  the  way 
they  are,  we  may  eventually 
get  to  the  point  where  we 
have  a  modern  day  Calvin 
Coolidge  suggesting  that  we 
only  buy  one  airplane  and  let 
everyone  take  turns."  Well, 

I'll  go  General  Slay  one 
better;  I'll  say  that  if 
things  don't  improve  by  the 
year  2000,  we  will  only  be 


These  rising  costs  of  systems 
makes  the  cost  credibility  of 
our  industry  more  and  more  sus¬ 
pect.  Despite  this  negative  pall 
that  hangs  over  us ,  everyone  in 
this  room  is  striving  to  improve 
our  performance  in  mission  capa¬ 
bility,  reliability,  maintaina¬ 
bility,  and  with  lower  cost.  What 
we  face  today  that  is  unlike  any¬ 
thing  we  have  faced  since  World 
VJar  II  is  that  we  are  attempting 
major  programs  in  the  face  of 
severe  resource  limitations. 

These  resource  limitations  apply 
to  the  government  as  well  as 
industry . 

I  would  like  to  briefly  describe 
some  of  these  resource  limitations-- 
some  which  are  familiar  and  some 
you  may  not  have  focused  on.  All 
have  or  will  have  an  impact  on  the 
way  we  do  business,  our  costs  and 
our  cost  credibility. 

Let's  take  the  easy  ones  first';  the 
government's  limited  resource  in 
dollars  and  thus  in  program  oppor¬ 
tunities.  If  we  stay  close  to 
home,  let's  look  at  Space  Division's 
constant  dollars  and  major  program 
starts  in  Figure  3.  When  we  re¬ 
member  that  space  is  barely  20 
years  old  this  is  hardly  an  exploding 
field  of  opportunities.  One  major 
program  start  every  three  years 
and  a  near  constant  dollar  budget 
despite  the  fact  that  space  com¬ 
munications,  surveillance  and 
defense  are  all  relatively  new  and 
growing  fields  of  technology, 
application  and  exploitation.  Cost 
credibility  is  vital  in  the  face  of 
this  limitation  because  without 
proper  planning  for  the  dollar 
resources  there  will  be  no  system 
to  work  our  mission  assurance  on. 

Part  of  the  answer  lies  in  the 
judicious  application  of  fixed 
price  or  incentive  contracts  but 
the  fallacy  here  is  the  assumption 
that  F.P.  or  F.P.I.  ensures  cost 
credibility.  The  government  often 
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views  (Figure  4)  cost  credibility 
on  a  sharply  falling  scale  with 
contract  type,  while  industry 
often  sees  little  difference  in  its 
ability  to  project  actual  costs. 

The  real  driving  force  in  cost  is 
not  the  contract  type  but  the 
degree  of  definition  and  the 
stability  of  firm  requirements. 

So  X"jhat ,  say  the  government  con¬ 
tracting  of ficers--f ixed  price 
is  still  fixed  price  to  the 
government  no  matter  what  the 
contractor's  cost.  Not  so, 
there  are  many  ways  to  skin  the 
cat  and  in  industries'  view 
there  are  some  standard 
approaches  to  v/orking  unex¬ 
pected  cost  problems  under  the 
type  of  contract  employed,  as 
shown  in  Figure  5.  The  records 
are  full  of  fixed  price  con¬ 
tracts  that  had  major  cost 
grov/th  from  Navy  ships  to  Air 
Force  aircraft. 

The  wall  between  the  govern¬ 
ment  and  the  contractor  is 
there  to  represent  the  ability 
of  the  contractor  to  obtain 
changes ,  waivers  or  devi¬ 
ations  to  the  contract  in 
order  to  help  solve  cost 
problems.  This  wall  or  the 
inability  to  waiver  or  change 
the  fixed  price  contract  poses 
the  greatest  threat  or  risk  to 
the  contractor  for  millions  can 
be  spent  on  insignificant  per¬ 
formance  parameters .  That  was 
one  reason  the  incentive 
contract  was  structured,  de¬ 
vised  to  give  some  flexibility 
on  both  sides.  The  trouble 
with  most  incentive  contracts 
is  that  they  revert  to  either 
fixed  price  or  cost  type  under 
the  strain  of  major  cost 
problems  since  the  share  lines 
are  normally  negotiated  to 
handle  so  called  "normal 
problems."  The  message  here  is 
simple,  the  drive  for  steeper  share 
lines  does  not  necessarily  motivate 
the  contractor  under  severe  cost 
problems.  Finally,  we  come  to  the 
old,  much  aligned  cost  olus  fixed 
fee.  Let  me  tell  you  from  years  of 
experience  that  there  is  no  other 


contract  type  where  the  government 
and  industry  work  closer  together 
to  solve  cost  problems. 

There  is  another  government  short¬ 
age  which  is  not  so  obvious  but 
which  can  have  major  impacts  on 
our  cost  credible  future- -man¬ 
power  shortages  in  the  officer  and 
civilian  support  side  of  the  house. 
Figure  6.  Believe  me,  there  is 
nothing  so  frustrating  or  more 
costly  to  a  program  than  a  govern¬ 
ment  project  office  that  is  so 
understaffed  that  timely  decisions 
are  impossible.  This  viewgraph 
gives  a  quick  look  forward  and 
backward  at  the  officer  level 
problem  in  AFSC.  Space  Division 
has  a  compounded  problem  that  all  of 
us  in  the  Los  Angeles  area  face-- 
high  property  values.  For  a  junior 
grade  officer  to  be  able  to  afford 
a  house  within  25  miles  of  the 
Space  Division,  he  would  have  to 
be  moonlighting  with  two  or  three 
extra  jobs.  Other  than  government 
housing,  I  know  of  no  solution  to 
their  problem. 

Let's  take  a  look  at  the  industry 
side  of  the  ledger  in  terms  of  its 
resources.  Manpower,  no  oversupply 
since  the  disastrous  slump  in  the 
early  70' s.  In  fact,  as  Figure  7 
shows,  we  hit  a  20  year  high  in  78 
and  continued  that  growth  in  79. 

The  oainful  slump  after  the  Vietnam 
war  is  evident  and  we  are  still 
suffering  today  from  its  effects. 
Perhaps  the  best  indicator  of  that 
shortage  is  the  degree  to  which  we 
are  competing  for  them  as  shown  by 
the  advertising  index,  Figure  8. 

VJe  are  scrambling  for  them  with 
five  times  the  intensity  we  did  10 
years  ago,  and  the  actual  employ¬ 
ment  of  over  400,000  in  R&D  exceeds 
by  almost  107e,  the  numbers  that  were 
employed  in  the  late  60' s.  Engi¬ 
neering  graduates  are  expected  to 
average  50,000/year  through  the  1985 
time  period  with  an  anticipated 
demand  of  over  56,000/year.  There 
appears  to  be  no  short  term  solution 
other  than  a  major  depression  and 
none  of  us  want  that  to  occur. 


Oil  is  not  the  only  scarce  com¬ 
modity  in  this  country  since 
1974.  Let's  take  a  look  at  one 
critical  material,  beryllium. 
Figure  9.  Pure  beryllium,  is  now 
processed  by  only  one  refiner, 
Brush-Vfellman  in  Pennsylvania. 
Lead  times  can  run  10  to  14 
months  and  for  special  titanium, 
24  months  has  been  quoted. 

Electronic  components  and 
especially  the  high  reliability 
types  that  we  need  for  our 
business  have  always 'been  a 
problem  for  our  industry. 

The  reason,  of  course,  is 
obvious,  small  quantities  with 
extreme  quality  requirements. 

Our  ability  to  get  the  inter¬ 
est  of  electronic  components 
industry  has  been  declining 
over  the  last  20  years  and 
commercial  sales  are  the 
reason  why  (Figure  10) .  The 
data  was  only  available 
through  76  on  this  partic¬ 
ular  study  and  the  situation 
today  is  even  worse.  The 
large  dollar  market  now  and 
in  the  future  will  lie  in 
commercial  sales  and  more  and 
more  component  manufacturers 
will  turn  their  backs  on  low 
volume,  high  reliability 
parts.  This  loading  of  the 
industry  reflects  directly 
in  lead  times.  From  a  study 
done  within  Hughes  (Figure  11) , 
lead  times  for  various  com¬ 
ponents  have  grown  over  the 
last  seven  years  by  factors 
of  two  to  four  and  no  parts 
are  unaffected.  These  lead 
times  are  only  averages  and 
do  not  include  hybrids  which 
have  been  running  12  to  14 
months  lately.  Of  course, 
everyone  has  their  own  horror 
story  here,  and  many  specific 
examples  of  18  months  and 
even  two  years  can  be  quoted. 

The  newest  limited  resource  that 
has  begun  to  impact  us  in  the 
last  five  years  is  available 
capital.  The  scarcity  of 
capital  is  reflected  in  the 
prime  commercial  bank  loan 
rate.  Figure  12.  Prior  to  1979, 


no  one  except  controllers  paid 
much  attention  to  prime  rates. 

Now  that  20%  has  been  reached  and 
we  have  all  experienced  the  credit 
crunch,  most  of  us  are  familiar 
with  prime  rates .  Primes  have 
backed  off  about  1%,  since  the  all 
time  high  of  20%,  in  March,  but 
most  financial  people  feel  that 
low  primes  under  10%  are  gone  for 
some  time  to  come. 

So  how  does  that  impact  our  business? 
Well,  for  one  thing  the  desire  to 
pursue  more  fixed  price  contracts  on 
the  government's  side  v/ill  meet  with 
increasing  resistance  from  industry 
unless  some  existing  policies  are 
modified. 

All  businesses  require  investments  in 
facilities,  machinery  and  equipment, 
and  working  capital.  The  defense 
business  has  traditionally  required 
less  investment  in  working  capital 
than  commercial  businesses  because 
the  government  provides  a  portion  of 
the  working  capital  investment 
through  progress  payments.  In  recog¬ 
nition  of  this,  defense  contracting 
profit  margins  on  cost  or  sales  are 
substantially  lower  than  those  of 
commercial  businesses.  Progress  pay¬ 
ments  at  an  80%,  rate  do  not  provide 
full  financing  of  the  contractor's 
working  capital  requirements  and,  in 
fact,  would  not  even  at  a  100%  rate. 
On  the  average,  80%,  progress  pay¬ 
ments  finance  approximately  607o  of 
the  contractor's  inventories  and 
receivables  and  he  must  finance  the 
balance.  This  difference  is  caused 
by  a  number  of  factors,  most  of 
which  are  determined  by  government 
procurement  policies.  First,  certain 
costs  such  as  vendor  and  subcontract 
costs  are  not  billable  for  progress 
payment  purposes  until  they  are  paid 
and,  therefore,  vendor  liabilities 
are  not  a  net  source  of  contractor 
financing.  Second,  progress  pay¬ 
ments  do  not  recognize  unallowable 
costs.  This  reduces  the  80%  rate  to 
an  effective  rate  of  less  than  78%. 

Third,  the  progress  payment  system 
has  several  built-in  lags  which 
result  in  the  receipt  of  progress 
payments  lagging  substantially 
beyond  the  time  when  the  con¬ 
tractor  must  pay  his  bills. 
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The  current  normal  807o  progress 
payment  rate  was  established 
at  a  time  when  the  prime 
interest  rate  operated  in  the 
6  to  87o  range.  Even  then,  it 
took  about  3 . 87,  of  negotiated 
profits  to  pay  for  the  interest 
and  other  unallowable  costs  and 
to  break  even  on  a  typical  12- 
month  fixed-price  production 
contract.  Now,  with  the  prime 
interest  rate  operating  in  the 
207o  range,  it  takes  about 
7.57o  of  negotiated  profits  to 
break  even  on  the  same  con¬ 
tract.  In  order  to  keep 
today's  contractor  in  the 
same  profit  position  that  he 
was  in  when  interest  costs 
were  in  the  6  to  87o  range, 
contract  profits  would  have  to 
be  increased  by  about  3 . 77o  or, 
alternatively,  the  normal 
progress  payment  rate  v/ould 
have  to  be  increased,  from 
807o  to  1007o  of  incurred 
cost . 

This  is  shown  graphically 
on  the  next  two  figures 
(13  and  lA) ,  first  realized 
profit  as  a  function  of 
borrowing  rate.  You  will 
notice  that  the  extra  one- 
half  month  lag  in  progress 
payments  that  many  of  us 
are  realizing  now  costs  us 
17o  point  in  fee.  It's 
obvious  why  the  accounts 
receivable  people  are  a 
pain  in  the  neck  these  days. 

The  second  graph,  Figure  14, 
presents  the  progress  pay¬ 
ment  necessary  to  hold  a 
constant  fee  rate  at 
various  borrowing  rates . 

You  will  notice  that  with 
a  1.5  month  lag ,  1007o 
progress  payments  are 
required  at  a  157,  borrowing 
rate  and  it  has  been  many 
months  since  any  of  us  have 
seen  that  rate. 

Let  me  give  you  another  example  of 
why  so  many  companies  shy  away  from 
government  fixed  price  contracts , 
Figure  15.  This  is  a  cash  flow 
comparison  of  a  typical  100  million 


dollar  program  of  four  years  duration 
A  comparison  is  made  of  a  107o  CPFF 
versus  a  157,  fixed  price  and  you 
will  notice  that  when  interest 
earned  is  considered  only  a  17<,  dif¬ 
ference  results.  We  have  not  been 
too  successful  in  negotiating  157o 
fees  even  on  fixed  price  contracts, 
so  I  have  also  plotted  the  EDSG 
experience  over  the  last  seven 
years  on  fixed  price  contracts 
which  average  out  at  4.37o.  VJhen 
the  interest  paid  is  considered  on 
this  experience  curve,  you  will  note 
than  an  additional  three  million  is 
lost  from  the  fee  at  a  207,  bor- 
rov/ing  rate  or  an  effective  fee  rate 
of  1.37.  Not  exactly  a  smashing 
business . 

So  in  the  face  of  these  problems, 
how  do  we  maintain  cost  credibility? 
Well,  we  have  two  major  areas  to 
address,  proposal  up  to  competitive 
award,  and  post  award  or  on-going 
contracts . 

Let's  take  a  look  at  the  pre-award 
cycle.  Figure  16.  The  normal  pro¬ 
gram  starts  24  months  before  the 
government  fiscal  year  in  which 
actual  dollars  would  be  appropriated 
and  36  months  before  any  program 
start.  Fiscal  guidance  is  estab¬ 
lished  by  the  Secretary  of  Defense 
at  -16  months,  the  services  response 
in  the  form  of  POM  at  -14  months 
and  so  forth.  A  long  and  torturous 
cycle  is  involved  and  many  inter¬ 
actions  between  the  services ,  the 
Secretary  of  Defense,  the  budget 
bureau  and  the  Congress. 

^’hat  is  often  the  result  is  a  built 
in  funding  problem,  as  shown  in 
Figure  17.  Government  and  industry 
take  too  little  time  in  clearly 
defining  the  program  and  cost  at  the 
early  phase.  How  many  of  you  have 
asked  or  been  asked  to  supply  rough 
order  of  magnitude  numbers  by 
Monday  of  next  week?  As  a  result, 
insufficient  funds  are  requested  and 
backed  into  the  budget .  The  long 
interval  between  POM  budget  sub¬ 
mission  and  RFQ  allows  another 
phenomenon  to  take  place--addi- 
tional  system  definition  and 
inflation.  This  cycle  leads  to 
the  ultimate  cost  growth  problem. 


The  government ' s  answers  to 
these  problems  have  been  design- 
to-cost  programs,  pre-RFP 
release,  and  fixed  price  and 
incentive  programs.  More 
should  be  done  by  both  govern¬ 
ment  and  industry.  First, 
industry  should  be  requested 
to  participate  more  often  in 
the  cycle  and  should  insist  on 
adequate  time  for  response. 
Second,  more  requests  should  be 
made  for  a  spread  of  costs 
niraibers  reflecting  our  f  :i- 
mate  of  the  degree  of  up.- ;r- 
tainty  in  the  program 
requirements,  technology  and 
schedule  achievability .  In 
this  regard,  more  use  should 
be  made  of  cost  models  but 
improvements  or  at  least 
sophistication  in  their  use 
is  required.  VThat  we  need 
is  a  good  Monte  Carlo  cost 
analysis  program  that  can 
account  for  the  variables 
of  design  firmness,  likely 
program  funding,  inflation 
and  interest  rates,  com¬ 
ponent  availability  and 
manpower  uncertainty. 

Third,  we  must  both  be 
mature  enough  to  accept 
the  fact  that  the  program 
may  not  be  important  enough 
to  our  country  to  tolerate 
the  reasonable  expectancy 
of  cost. 

In  the  post  award  period, 
both  industry  and  government 
have  roles  and  responsibility 
to  maintain  cost  credibility 
(Figure  18) .  Industry  cer¬ 
tainly  must  be  willing  to 
adopt  early  problem  warning 
systems  such  as  earned 
value  and  to  conduct  their 
business  where  dollars  and 
schedules  are  as  paramount 
as  the  performance  of  their 
products.  Government  in 
turn  must  define  its 
program  needs,  establish 
firm  funding  profiles  or 
multi-year  funding.  As  a 
minimum,  we  should  both 
have  the  courage  to  admit 
that  the  risk,  technical  or 
funding  is  too  great  to 


proceed  into  high  expenditure  phase 
of  the  program. 

In  summary,  let  me  say  that  our 
resource  limitations  are  real.  Ue 
do  have  declining  real-dollar  Space 
Defense  budgets ,  fewer  new  program 
starts,  declining  military  experi¬ 
ence  level,  industry  engineering, 
material  and  component  shortages. 

We  are  experiencing  high  inflation 
and  interest  rates,  and  the  general 
business  environment  seems  to  be 
increasingly  unstable. 

But  all  is  not  doom  and  gloom, 
particularly  with  respect  to  cost 
credibility.  General  Slay's  new 
initiatives  for  multi-year  procure¬ 
ments,  selection  on  the  basis  of 
past  performance,  draft  RFP ' s  and 
the  desire  to  get  the  Air  Force 
procurements  on  a  more  commercially 
oriented  basis  are  all  steps  in  the 
right  direction.  I  believe  that 
limited  resources  and  full  capacity 
will  aid  the  government  in  obtaining 
cost  credible  proposals  since  indus¬ 
try  will  be  looking  carefully  at  the 
risk  versus  the  profit  gain.  Finally, 
I  believe  the  government  will 
recognize  the  rising  cost  of  capital 
and  the  growing  capital  equipment 
intensity  of  our  industry.  This 
recognition  V7ill  force  the  govern¬ 
ment  to  improve  the  incentives 
offered  in  order  for  industry  to 
enter  into  fixed  price  contracts. 
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AVERAGE  VEAHLV  PRODUCTION  HATE  I  I  COST.  THOUSANDS  OF  DOLLARS 


MU 


IMPACT  OF  36-MONTH 
BUDGET  CYCLE  ON  COST  CREDIBILITY 


ESTABLISHING  PROGRAM  COST 
CREDIBILITY  AFTER  AWARD 
INDUSTRY  AND  GOVERNMENT 
RESPONSIBILITIES 

•  INDUSTRY 

•  EARLY  PROBLEM  WARNING  SYSTEMS 

•  EARNED  VALUE 

•  DEFINE  COST  AND  SCHEDULE  RESPONSIBILITY  AT  A  LEVEL  WHERE 
THE  DOLLARS  ARE  SPENT 

•  CREATE  A  WORKING  ENVIRONMENT  IN  WHICH  IT  IS  FELT  PROBLEMS 
CAN  BE  EXPOSED 

•  MUST  DEVELOP  REPORTING  FORMATS  THAT 

•  IDENTIFY  THE  KEY  PARAMETERS 

•  SHOW  HISTORY 

•  CORRELATE  PROBLEMS  TO  COST  AND  SCHEDULE  DEVIATIONS 

•  BASE  SALARY  REVIEWS  OF  COST  AND  SCHEDULE  PERFORMANCE  NOT 
JUST  TECHNICAL  PERFORMANCE 

•  DEVELOP  DATA  BASES  ON  PAST  PROGRAMS  TO  BE  UTILIZED  DURING  BIDDING 
•GOVERNMENT 

•  BETTER  DEFINITION  OF  PROGRAMS  LESS  REDIRECTION) 

•  RECOGNIZE  THE  DIFFERENCES  IN  COST  VARIABILITY  OF  DEVELOPMENT 
VERSUS  PRODUCTION  PROGRAMS 

•  MINIMIZE  THE  FISCAL  YEAR  FUNDING  CONSTRAINTS 

•  COST  VERSUS  BENEFIT  TRADEOFFS  OF  GOVERNMENT  CONTRACTOR 
SUPERVISION  VERSUS  LAISSEZ-FAIRE 


MANAGEMENT  OF  PROGRAMS 

Don  Hurta 
Naval  War  College 

The  Concepts  and  Tools  of  Risk  Management 

The  objective  of  this  workshop  is  to 
discuss  the  application  of  risk  metho¬ 
dology  to  the  function  of  Program 
Management.  A  short  presentation  will 
be  made  on  the  concepts  associated  with 
risk  analysis,  risk  assessment  and  risk 
management.  The  workshop  will  then 
discuss  the  potential  application  of 
these  concepts  as  a  program  management 
tool  during  the  program  acquisition  and 
the  program  implementation  phases. 
Recommendations  will  be  made  as  to  where 
risk  analysis  and  assessment  should  be 
applied  and  how  risk  mangement  can  be 
used  to  support  effective  program 
implementation  and  communication. 
Particular  emphasis  will  be  placed  on  the 
potential  characterization  of  the  risk 
as  an  inherent  element  of  schedule,  cost 
and  performance  management. 

A  set  of  analytical  tools  is  applied 
to  quantify  risk  as  a  function  of 
variables  of  probability  of  success  and 
and  consequence  of  failure.  It  is  main¬ 
tained  that  the  characterization  of  a 
system  model,  coupled  with  the  use  of 
risk  analysis  tools  will  provide  manage¬ 
ment  insight  into  technical,  cost  and 
schedule  trade  offs,  ECP  management, 
cost  sensitivity  analysis  and  decision¬ 
making. 

Basic  concepts  and  tools  of  risk  analysis 
will  be  presented  to  the  splinter  workshop 
group,  including  the  risk  function, 
risk  modeling  techniques,  and  the  develop¬ 
ment  of  system  models.  Discussions  will 
be  held  to  evaluate  the  follwoing 
questions: 

1)  Given  program  funding  and 
schedule  constraints,  can  this 
technique  be  used  to  assist 
the  program  manager  in  meeting 
performance  objectives? 

2)  Where  can  the  risk  management 
techniques  be  best  used  to 
enhance  the  attainment  of  mission 
assurance  goals? 


3)  Can  this  technique  be  used  to 
emphasize  the  need  to  be 
responsible  to  program  goals, 
rather  than  dictate  program 
requirements  (eg,  A109)? 

4)  How  effective  are  these 
techniques  in  supporting 
trade-offs  analyses  and  pre¬ 
venting  "goldplating"? 

5)  Do  the  techniques  of  risk 
management  provide  objectivity 
to  the  establishment  of  minimum 
acceptable  standards  for  mission 
assurance? 

6)  Do  risk  management  techniques 
apply  to  both  government  and 

industry?  . to  prime 

contractors  and  subcontractors? 

If  so. . .how? 

7)  Who  is  responsible  in  the 
program  organization  for  risk 
management? 

8)  How  effective  is  risk  management 
as  a  communications  vehicle? 

9)  Can  risk  management  techniques 
be  used  to  establish  contract 
incentives? 

10)  Is  there  a  danger  in  using  these 
tools  too  extensively? 

11)  What  are  the  dangers  in  treating 
the  elements  of  the  system 
model  independently? 

12)  Where  does  judgment  and  decision¬ 
making  enter  into  the  process 
and  analysis  end? 

"WHEN  I  USE  A  WORD,"  HUMPTY  DUMPTY 
SAID,  IN  RATHER  A  SCORNFUL  TONE, "IT  MEANS 
JUST  WHAT  I  CHOOSE  IT  TO  MEAN  -  NEITHER 
MORE  NOR  LESS." 

"THE  QUESTION  IS,"  SAID  ALICE, "WHETHER 
YOU  CAN  MAKE  WORDS  MEAN  SO  MANY 
DIFFERENT  THINGS." 

"THE  QUESTION  IS,"  SAID  HUMPTY  DUMPTY, 
"WHICH  IS  TO  BE  MASTER  —  THAT’S  ALL." 

THROUGH  THE  LOOKING-GLASS 


LEWIS  CARROLL 


RISK 


PROBABILITY 

OF 

FAILURE 


CONSEQUENCE 

OF 

FAILURE 


AGAINST  WHAT  STANDARD  ? 
WHAT  IS  THE  GOAL  ? 

WHAT  IS  MINIMALLY 
ACCEPTABLE  ? 


RISK  MODELS 

RISK  PROFILES 
UTILITY 

MULTI-AHRIBUTE  UTILITY 
ISO-RISK  CONTOURS 
NETWORKS 
(MODIFIED)  DELPHI 


D0U6LAS 

MacARTHUR'S 

CREDO 


EVERY  MISTAKE  IN  WAR  IS 
EXCUSABLE  EXCEPT  INACTIVITY 
AND  A  REFUSAL  TO  TAKE  RISKS. 

...SAID  BY  H.H  FROST 

...  QUOTED  IN  AMERICAN 
CEASAR 


NO  RISK  18  THE  HIGHEST  RISK  OF  ALL 

AARON  WILDAVSKY 

IN  ORDER  TO  WIN  YOU  MUST  RISK  LOSS 

JEAN-CLAUDE  KILLY 


1  >0 

|l*»MOOOCl 

ICC  ktoDuciin.nT 


>  TIITl  [BTIFj  \K\  TiBt  |  |  MF,  |  |  QTUFC  |  [mst| 


•M4  IHS 

I*'  MOO  KL. 


ISO  ISO  HO  so  T  I  S  IS  li  IT  tl  (>  SO 
MTIF.IOURS  KSFOBSC  TMLKC  OTHK.IM 


tm  tm 

HP 


TARGET 

DESTRUCTION 


SENSITIVITY  ANALYSIS,  WEIGHTS 


T5&co«t,k^ 


MTBF,  HOURS 


20  IS  10  2  0  U 


COST  o.A  — 

b.B  >> 
e.C  — 


BASELINE 

MAXIMALLY  DESIRED 
MINIMALLY  ACCEPTABLE 


VALUC 


WCtOHTS 

8 

c 

errecTivENess  so 

FT 

*• 
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COST  SO 

♦  T 
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WEiOnTCOvAluC 
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tso 

sooi 

OOU0L€ 

CYYCCTlVCNeSS  WEIGHT 

CffCCTlVCNESS  ICO 

1  *1 

1  '®l 

COST  so 

u 

12 

WEtGMTEO  VALUE 

Cm 

■ISO' 

EFFECTIVENESS  WEIGHT 

EFFECTIVENESS  IS  I 

1  ’1 

[  '1 

1 

,  COST  SO  1 

WE lOHTEO  VALUE  ^ 

^MS 

ra 

COST  WCiGHT 


. MEASURING  GOST  EFFECTIVENESS  (IS)  OF 

LIMITED  USE  IN  STRATEGIC  PLANNING  BECAUSE 
THE  RANGE  OF  OPTIONS  IS  ACTUALLY  RESTRICTED 
BY  POLITICS  MOREOVER  THE  SEEMING  PRECI¬ 
SION  OF  COST  EFFECTIVENESS  (GETS)  IN  THE  WAY 
OF  REAL  UNDERSTANDING  OF  THE  MOST  DIFFICULT 
PROBLEMS  IN  DEFENSE  WHICH  INVOLVE  VALUE 
JUDGEMENTS  ' 


CONTINGENCY 


lAMFS  SCHIESINGER 


SUMMARY  OF  MULTI-ATTRIBUTE  UTILITY  LEARNING  OBJECTIVES 


ATTRIBUTES 

inocpcnocnt 

EXHAUSTIVE 

VVEIGHTS 

relative 

additive 

utility  functions 

SHAPES  LIMITED  ONLY  BY  INGENUITY 

REFLECT  WEIGHTS 

BASELINE 

MINIMALLY  ACCEPTABLE 
MAXIMALLY  DESIRED 

ALTERNATIVE  CHOICE  CONSIDERATIONS 

PROPER  INPUTS 

DECISION  MAKER  •  THE  MORE  PARTICIPATION  THE  BETTER 
experts  -  WITH  RATIONALE 
DISCRIMINATION  CAPABILITY 
model  LIMITED 
NON -QUANTIFIED  FACTORS 
REQUIREMENTS 
SENSITIVITY  ANALYSES 

DO  ONE.  .  .  . 


RISK 


100%  0% 


0% 


100% 


STS  PAYLOAD  INTEGRATION 

Leonard  Nicholson 
Johnson  Space  Center 


STS/PAYLOAD  INTEGRATION  PROCESS 


I- 


ftEOUMEWCMTS 

OEFINITIOMS 


PAYLOAD  INTEGRATION  PLAN 


•  DEFINE  PAYIOAD/STS  ROLES  AND  RESPONSIBILITIES 

•  DEFINE  TECHNICAL  BASELINE  FOR  IMPLEMENTATION 

•  ESTABLISH  GUIDaiNES  AND  CONSTRAINTS  FOR  INTEGRATION  AND 

PLANNING 

•  DEFINE  TASKS  TO  BE  ACCOMPLISHED 

•  ESTABLISH  SCHEDULE  FOR  ALL  INTEGRATION  AaiVITIES 

•  ESTABLISH  BASIS  FOR  DEFINITION  AND  PRICING  OF  OPTIONAL 

SERVICES 

m  ORGANIZATION  AND  SCOPE 

•  ORGANIZED  BY  MAX)R  DISCIPLINES  i.e.,  LEVa  I GROUNORULES/ 

RESPONSIBILITIES,  ENGINEERING,  aiGHT  OPERATIONS.  AND 
GROUND  OPERATIONS 

•  INCLUDES  ALL  STS  INTERFACES  WITH  THE  USER 


PAYLOAD  INTEGRATION  PLAN  ANNEXES 


STS/PAYLOAD  DCX^UMENTATION 


•  PAYtmO  INTEGRATION  PLAN 


•  PAYLOAD  INTEGRATION  PLAN  ANNEXES 


•  PAYLOAO/STS  ICO 


I  SAFETY  DOCUMENTATION 

•  HAZARD  REPORT  FORMS 

•  COMPLIANCE  REPORT 

•  HAZARDS  PROCEDURES 


•  DEFINE  ROLES  AND  RESPONSBILITIES 

•  DEFINE  TASKSAVORK  TO  6E  DONE 

•  ESTA0LISH  BASELINES,  GUIDELINES  AND 

CONSTRAINTS  FOR  INTEGRATION  ANO 

planning 

•  ESTABLISH  SCHEDULE  FOR  ALL  INTEGRA¬ 

TION  ACTIVITIES 

•  DOCUMENT  AGREED  UPON  PAYLOAO  IMPU- 

MENTING  PROCEDURES.  TIMELINES,  PRO¬ 
FILES,  ANO  PLANS  WHICH  STS  WILL  BE 
REQUIRED  TO  EXECUTE 

•  DOCUMENTATION  OF  DETAIL  DATA  LISTS  ANO 

REQUIREMENTS  NECESSARY  FOR  STS  TO 
BUILD  FLICHT/GROUNO  SOFTWARE  OR  TO 
CONFIGURATE  FLT/GNO  SYSTEMS  FOR 
IMPLEMENTATION  OF  PAYLOAO  MISSION 

•  DEFINE  DESIGN  ANO  SPECriCATIONS  FOR 

INTERFACES  BASELINED  IN  THE  PIP 

•  ESTABLISH  ENVIRONMENTS  FOR  DESIGN 

PURPOSES 

•  ESTABLISH  HAZARDS  TO  BE  CONTROLLED 

•  REACH  AGREEMENT  OF  METHODS  FOR 

hazard  CONTROL 

•  ESTABLISH  METHODS  FOR  VERIFYING  COM¬ 

PLIANCE  WITH  SAFETY  REQUIREMENTS 


%  FLIGHT  ^LANNlMi 

•  CMWACTIVITICSPLAN/TVCMT  SCQiCNCf 

•  POMC*.  ATTITUDE.  THERMAL  PROFilC 

•  TRAjCCTORIC&^LAUMCHWIWXMrS 

%  RAYLOAO  DATA  PACKAGE 

•  SEOUCRCEOMASSPtOPERTKS 

•  CONFICURATlOllDRAWtllCS 

•  RF  RADIATION  DATA 

%  POCCRCOUMCMCNTS 

•  VOL  1- JSC  ROCC  RtOUlREWCMTS 

•  VOL  a*  MCC  to  REMOTE  CENTER  INTERFACES 

#  COMMAND  ANO  DATA 

%  CREWCOMPARTkCNT  STOWAa^lNSTALLATION 

#  LAUNCH  SITE  SUPPORT  PLAN 

•  '*ROCESS  PLAN 

•  FACILITIES  AND  SERVICES 

•  CHECKOUT  PROCEDURES 


I  FllCMT  OPERATIONS  SUPPORT 

•  OPERATIONS  PLAN 

•  MISSION  RULES 

•  DATA  EXCHANCE  TIMELINES 

•  PROCEDURES 


•  UNDUE  VEtriCATION  RCOUMEMENTS 


STS/PAYLOAD  REQUIREMENTS  DERNITION 
DCXJUMENTATION 
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I 

FUTURE  CHALLENGES  FOR 
REUSABLE  SPACE  SYSTEMS 

28  April  1980 

Ben  Wier 

I  General  Dynamics 


ATLAS  E/F  LAUNCH  VEHICLE  PROGRAM 

•  BACKGROUND 

•  STORAGE 

•  REFURBISHMENT  AND  MODIFICATION 

•  PROGRAM  TIME  ADJUSTMENTS 

•  BRIEF  LOOK  AT  THE  80'S  AND  90'S 


BACKGROUND 


ATLAS  E/F  MODIFICATION  PROGRAM 


ATLAS  E/F  PROCESSING 

NORTON  AFB  (SAN  BERNARDINO) 

•  STORAGE 

•  PERIODIC  INSPECTION 

•  CORROSION  CONTROL 

•  SfflPMENT  TO  GDC  SAN  DIEGO 


•  OPERATIONAL  WEAPON  SYSTEM  IN  EARLY  60'a 

•  DECOMMISSIONED  IN  196S 

•  STORAGE  AT  NORTON  AIR  FORCE  BASE,  SAN  BERNARDINO 

•  FIRST  FLIGHT  -  1967 

•  FLOWN  TO  DATE 

•  25  REMAIN  TO  BE  FLOWN 

A  GLOBAL  POSITIONING  SYSTEM  -  PHASE  I 
A  GLOBAL  POSITIONING  SYSTEM  -  PHASE  II 
A  ABSAD  PROGRAM 
A  TIROS/NOAA  PROGRAM 

A  DEFENSE  METEOROLOGICAL  SATELUTE  PROGRAM 
A  THIRTY-FOUR  OUT  OF  35  SUCCESSFUL 


CURRENT  ATLAS  EfF  CONFIGURATIONS 


ATLAS  E/F  PROCESSING 

GENERAL  DYNAMICS  (SAN  DIEGO) 

•  WELD  84-INCH  TANK  RING  KIT 

•  FAB  BASEUNE  AND  MISSION  PECULIAR 
DETAILS/PARTS 

•  FAB  NEW  AUTOPILOTS 

•  FAB  FAIRINGS  AND  ADAPTERS 


ATLAS  E/F  PROCESSING 

GENERAL  DYNAMICS  (VAFB) 

•  VAMP 

•  FAB  BASEUNE  AND  MISSION  PECUUAR 
DETAILS/PARTS 

•  INSTALL  MISSION  PECULIAR  KITS 

•  LAUNCH  SITE  MAINTENANCE  AND  OPERATION 

•  LAUNCH  PROCESSING 
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VANDCNB£«G  ATLAS  MODinCATION  PROGRAM  (V*MP) 


RtClIVlM; 
INSi'K'TION 
4  Slum  DOWN 


A  WWT 


IRAN  4  M04I.  TO 

IKHMI.H 

BAStUNt 


AUi>  MI*»ION 
PCCULIAR 
MOO  KITS 


iupv 


1 11 


Ul'MiM  itTAO  lUUNCII  AastauCNT 

AIiniTION  C/U  4 

SYSTIMS  lAllNCH 

INTIOlUTLI) 

Ctlll'KtMJT 


VANOENBCRG  atlas  modification  program  (VAMPt 


•  ALL  TASKS  ACCOMPUSHED  IN  BOOSTER  ASSEMBLY  BUILDING  NO.  6  (BAB-A) 

•  BOOSTER  RECEIPT  AND  INSPECTION 

4  (NVENTORY  BOOSTER  AND  tOOSE  EQUIPMEKT 

4  ACCOMPLISH  SHAKEDOWN  TO  GOC«BRWflT-001  DOCUMENT  ALL  AREAS  OP  BOOSTER 
INCLUDING  INTERNAL  PUEL  A  LO}  TANKS  PLUS  ALL  LOOSE  EQUIPMENT. 

4  REVIEW  ALL  BOOSTER  HITTORY  DOCUMENTATION. 

4  DISCREPANCIES  DOCUMENTED  ON  QUAUTY  ASSURANCE  FORMS  7000  (QAR) 

AND  CATEGORIZED  FOR  BNRB  ACTUH. 

•  BOOSTER  HARDWARE  REVIEW  BOARD  (BHRB) 

4  APSOaV  CHAIRMAN.  CONVAIR.  SAMTO  (SMSTH  TEST  WING  AND  APQC). 

ASSOCIATE  <^A>NTRACTORS 

4  APPROVE  OISPOStTION  OP  DISCREPANCIES 

4  BOOSTER  MODIFICATION  REPAIR 

4  ACCOMPUSH  BASELINE  MODS  IN  ACCORDANCE  WITH  a  SPEC.  27-003M  AND  ALL 
OUTSTANDING  SPEaPICATPON  CHANGE  NOTICES  (SCNs). 

4  AC(T>MPIJ8H  IRAN  TASKS  IN  ACCORDANCE  WITH  QAR  USPOSITIONS. 

■  COMPONENT  RECYCLE.  REVERiriCATXJN 

4  ACCOMPliSH  CRITICAL  COMPONENT  REVERIFICATION  IN  ACCORDANCE  WITH 
ATOSE.  F  PAKAMETKK6  DOCUMENT  AND  MISSION  ASSURANCE  PLAN  <MAP) 

4  RECYCLE  ENiilNES  IN  ACCORDANCE  WITH  RCN.'KETUYNE  DOCUMENTATION 


PERFORM  MISSION  PFCUlJAK  MODIFICATIONS 


BOOSTER  CHECKOUT  AND  ACCF; PTANCF 

•  CHECKOUT  IN  ACCtJRDANCi:  WITH  a  SPEC  27.003W 

4  HAKIAVARE  AND  rOCUMENTATION  REVIEW  BY  AFSD  LV.  AEROSPACE.  ANOSAMTO 
4  DD-230  ACCOMPIJSHED 


TIME  AND  HARDWARE 


4  ATLAS  MA-3  ENGINE  SYSTEM 
4  BUILT  IN  EARLY  60’S 
4  LAST  HOT  FIRED  IN  EARLY  60'S 

4  CONnGURATION  VERIFICATION 

4  SOME  DISASSEMBLY  -  HOROSCOPE  CHECKS 

4  EXTENSIVE  CONTAMINATION  CHECKS 

4  REVIEW  OF  PAST  ACCEPTANCE  DATA 

4  NO  SIGNIFICANT  REJECTS  AFTER  20  YEARS 

4  SAME  FINDINGS  APPLY  TO 
4  PROPELLANT  TANKS 
▲  THRUST  STRUCTURE 


TIME  AND  SPECIFICATIONS 

•  LONG  TERM  PROGRAMS  (10  YEARS  PLUS)  ADJUST  TO  CHANGING 
REQUIREMENTS,  E.  G., 

A  ENVIRONMENTAL  REQUIREMENTS 

-  VIBRATION  -  SINE/RANDOM/ ALL  RANDOM 

-  TEMPERATURE  CYCUNG 

-  ACOUSTIC  REQUIREMENTS 

A  ELECTRONIC  PIECE  PARTS  UPDATE 

•  AFFECTS  PREVIOUSLY  QUAUnED  HARDWARE 

A  QUALIFY  BY  ANALYSIS 
A  REQUAUFY 


TIME  AND  LOGISTICS 


•  HARDWARE  VENDOR  CHANCES  OCCUR  DURING  SPAN  OF  THE  PROGRAM 
A  BUSNESS  BASE  CHANCES 


A  OUT  OF  BUSINESS 


SELECT  NEW  SOURCE 

A  HARDWARE  REQUAUFICAT10N 


IN  THE  80'S  AND  EARLY  90'S 


•  ORBITAL  TRANSFER  VEHICLE 


•  PLATFORMS  IN  SPACE 
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QTV  DEVELOPMENT  PLAN 


Key  OTV  technologies 
Spece  cryogenics 
ToroWsI  tanks 
Aero  brake 

A<^anced  avionics  ^ 
Advar>ced  engme 


Engine  developmeni 


Study  Phase  A 


I  ^Phase  E 


|-  h  r  : 

^  I  I 


IpJ*  ACCElilAllgtl  OF  lA»Ct  SPACE  SySTEMS  IS 

APPUCABLI  TO  CaOPLATFOeii  A  IPACB  KADAR 


1  Lamtt  OTV  a  IoMmI  teececflt 


i  M  L£0.  O&fjlirt  S  chacA  out 
Can  adMi  or  ropar  by  EVA 


yloadloOEOby 
a  avusi  OTV 


1980  19A2  1984  1986  1988  1990 


>  Large  space  structure  sue  oottmued  el  low  ecceieration  «  0  05  to  0  3g  maw 

•  Optmnum  thrust  ■■  1 .000  to  3.000  l>  with  1  S.OOO  b  spacecralt 
Up  to  9  burrw  during  1  -day  trar^fer  to  GEO 

•  Low  thrust  can  be  achieved  by  running  1 SK  engir>e  at  idle  or  usmg  new  smal  c 


A  LEADING  OTV  CANDIDATE  CONCEPT 


LAIICE  SPACE  SYSTEMS  AIIE 


55,000  lb  maximum  prociellant  caE>acily 

Toroidal  tank  to  reduce  length 

Modified  RL1 0  engine  with  idto  mode 

Advanced  avionics  with  return-trip  capability 

Cryogenic  interfaces  with  Orbiter  &  KSC  per  STS  Centaur  study 

Deployable  aero  brake  kit  for  return  to  Ofbiter 


30*ft  payload 


&p»e«  baaad  radar 


Qao  platform 


/•77 

'"/7 


•  Hexagon  -  circular  •  Retar^uiar  >  hexagon  •  Long  ractangla 

•Central  mast  -f  spokes  •  Many  mass  cortcentrations •  Even  mass  distribution 

•  Flexible  •  Relatively  stiff  •  Very  flexible 

•  Low  natural  frequency  •  High  natural  frequency  •  Very  low  natural  frequency 

Low  acceleration  Including  thruet  tranelants 
may  be  vital  to  large  spece  Byeteme 


AVIONICS  REQUIREMENTS  FOR  OTV 
10-day  round-trip  misslona 

•  Guidance  ufxlate  S  navigallan  (space  sextant  A  star  tracker) 

•  Light  fuel  cels  using  main  propelants 
SunrivabilHy 

•  Proleciton  from  solar  flux,  Van  Alen  belt 

•  Defense  against  ECM  or  lasers 
Improvad  radundancy  architaetura 

•  FauH-tolerant  computers  with  internal  cross-strapping,  memory  bar* 
sharing,  &  improv^  software  techniques 

DItIributsd  computer  arrangomania 

•  Fiber-optics  bus 

•  Very  high  speed  (>450,000  operations  Iter  second) 

Improved  rollablllly/manraHng 

•  FiMjIt-tolerance/backups 

•  Condition-monitored  mainlenance 

Advanced  avioniea  provMo  Improvod  capsbHIty 
tor  OTV  witiwut  obaolaacanc#  by  1990. 
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SUMMARY  OF  ISSUES 
AND  RECOMMENDATIONS 


MANAGEMENT  OF  PROGRAMS 


3.  Flow  Down  Requirements 


SUMMARY  OF 

ISSUES  AND  RECOMMENDATIONS 


1.  Government  Roles  and 
Responsibilities 

Issue 

Change  Control  -  To  what  level 
should  the  Government  be  involved 
in  change  control  procedures  of 
prime  and  sub-tier  contractors  ? 

Recommendations 

oMIL-STD-483  should  be  more 
explicit  in  the  ECP  preparation 
in  the  area  of  Mission  Assurance. 
(Contractors  should  be  required 
to  state  the  total  effect  on  the 
system  of  the  change  involved. ) 

o  Consideration  should  be  given  to 
tailoring  the  existing  standards  to 
suit  the  particular  contract. 

2.  Cost  Estimating 
Issue 

Cost  Estimating  and  Management  - 
How  can  the  Government  get  a  good 
cost  estimate  in  the  early  phases 
of  a  program  and  a  continued 
estimate  throughout  the  program 
when  changes  are  contemplated  ? 

Recommendations 

o  Expand  existing  cost  modeling 
methods  and  develop  improved 
methods  for  independent  develop¬ 
ment  of  cost  estimating. 

o  Early  involvement  of  contractors 
in  the  Government  planning  pro¬ 
cess  and  cost  modeling. 


Issue 

Contracts  and  sub-tier  flow  down  - 
Should  the  Government  require 
flow  down  and  if  so  to  what  level  ? 

Recommendations 

olt  is  the  Governments  responsi¬ 
bility  to  require  flow  down  on  a 
case  by  case  basis  depending 
upon  the  criticality  of  the  con¬ 
tract  item.  (Items  of  particular 
concern  are  FMEA,  failure 
analysis,  change  control,  etc. ' 

4.  Program  Office  Personnel 
Continuity 

Issue 

Government  involvement  in  details 

-  Some  Government  offices  have 
inadequate  personnel  to  get  in¬ 
volved  in  sufficient  detail  in  pro¬ 
gram  activities.  Early  exper¬ 
ienced  involvement  is  particularly 
important. 

Recommendations 

o  Consideration  should  be  given  to 
provide  a  sufficient  number  of 
civil  servants  positions  to  pro¬ 
vide  continuity  and  early  involve¬ 
ment  in  detailed  program  plan  - 
ning. 

5.  Contractor  Assistance 
Issue 

Providing  help  to  the  contractor 

-  Does  the  Government  have 

a  responsibility  to  provide  help  to 
a  contractor  and  what  are  the  con- 
tractural  problems  that  may  be 
encountered? 

Recommendations 

o  The  Government  has  a  respon¬ 
sibility  to  provide  competent 
help  to  a  contractor  in  needed 
areas.  Provisions  for  this 
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assistance  should  be  included  in 
contract  formulation. 

Up  Front  Funding 

Issue 

A  contractor  study  has  shown  that 
some  contracts  have  been  let  with 
insufficient  attention  and  resource 
allocation  to  prevent, ative  analyses 
such  as  FMEA,  single  point  failure 
sneak  circuit  analysis,  etc. 

Recommendations 

o Government  should  place  con- 
tractural  emphasis  and  enforce 
requirements  for  preventative 
analysis  by  contractors  and  pro¬ 
vide  sufficient  funding  and  sche¬ 
dule  to  do  an  adequate  job.  The 
Government  must  provide  ade¬ 
quate  resources  to  review  the 
contractors  analysis. 

o  Consideration  should  be  given  to 
include  these  analysis  as  a  part 
of  the  incentive  program  for 
Mission  Assurance. 

o  Industry  must  provide  earlier- 
more  realistic  cost  estimates. 

Tailoring  of  Specifications 

Issue 

In  some  agencies  inadequate  atten¬ 
tion  has  been  given  to  tailoring 
existing  specifications  to  suit  the 
specific  requirements  of  a  pro¬ 
gram. 

Recommendations 

o  Consideration  should  be  given  to 
providing  training  programs  to 
increase  awareness  of  Misf  .on 
Assurance  requirements  so  that 
existing  specifications  can  be 
properly  tailored  at  the  outset  of 
a  program. 


8.  Communication 
Issue 

There  appears  to  be  a  growing 
adversary  situation  between  the 
contractor  and  customer,  i.  e.  , 
both  Government  and  sub- contract¬ 
or,  at  both  the  management  and 
the  working  level.  It  could  be  a 
result  of  less  informal  team 
interrelationships;  perhaps  caused 
by  over  emphasis  of  arms  length 
relationship.  Information  flow 
seems  to  be  impeded. 

Recommendations 

o  Education  of  Governnment  tech¬ 
nical  project  officers  to  en¬ 
courage  personal  contacts  with 
contractor  counterparts  while  at 
the  same  time  maintaining  ethical 
standards. 

o  Motivation  to  instill  a  team,  goal 
oriented,  spirit  for  both  industry 
and  Government. 

o  Policy  statement  to  recognize 
common  Government  industry 
goals. 

9.  Indus  try  Responsibility 
Issue 

Higher  profits  in  a  fixed  price 
environment  with  higher  risks  to 
the  contractor  is  a  declared  A.  F. 
policy.  Is  industry  willing  to 
accept  the  challenge  in  the  face  of 
the  uncertain  economy  and  un¬ 
certain  cost  of  money  ? 

Recommendations 

o  Yes,  but  provisions  to  protect 
catastrophic  events  must  be 
afforded  to  the  contractor.  Con¬ 
tractors  are  willing  to  accept 
the  risks  that  fall  within  their 
ability  to  control  through  good 
management  techniques.  A  re¬ 
commendation  that  might  be  con¬ 
sidered  is  a  form  of  imputed 
coverage  for  working  capital 


along  the  lines  of  cost  of  money 
coverage  for  facilities  invest¬ 
ments,  and  to  further  technology. 

o Industry  should  educate  their 
Government  counterparts. 

10.  Industry  Responsibility 
Issue 

Review  of  draft  RFP's. 

Recommendations 

oAbsolutely  essential  industry 
responsibility  is  a  vigorous  co¬ 
operative  constructive  review  of 
draft  RFP's.  The  Air  Force  is 
more  and  more  providing  oppor¬ 
tunity  -  industry  must  take  ad¬ 
vantage. 

11.  Industry  Responsibility 
Issue 

Contract  forms  -  preception  exists 
that  the  Space  Division  is  base¬ 
lining  on  fixed  price  and  NASA  on 
CPAF. 

Recommendations 

o  Although  we  have  experienced 
variations  on  the  old  theme  -  FP 
-  incentive,  award,  etc.  ,  we 
must  become  more  innovative  and 
creative.  The  Air  Force  is  solic¬ 
iting  suggestions  and  industry 
should  be  responsive. 

o  Earlier  planning  to  identify  the 
appropriate  contract  vehicle. 

12.  Payload  Integration  Management 
Issue 

Too  many  organizations  in  the 
integration  loop  for  AF  missions 
on  shuttle.  In  the  commercial 
world  -  JSC  interfaces  directly 
with  S/C  program  manager. 


Recommendations 

oSD  review  integration  manage¬ 
ment  with  an  eye  to  shortening 
the  technical  and  management 
communication  loops. 

o  Identify  deployment  modes  and 
environmental  constraints. 

o  Identify  design  criteria  and 
assure  understanding  before 
spacecraft  design  begins. 

13.  Payload  Integration  Management 
Issue 

There  is  perception  in  the  shuttle 
program  -  that  each  user  has  the 
idea  that  the  shuttle  and  the  shuttle 
flight  is  dedicated  to  his  space¬ 
craft. 

Recommendations 

o  The  Mission  Assurance  forum 
helps  disseminate  this  informa¬ 
tion  and  clarify  the  mixed  mani¬ 
fest. 

oSPO's  should  insure  that  space¬ 
craft  contractors  comply  with 
mixed  manifest  concept  and  con¬ 
straints  -  possible  communication 
suggestions; 
o  Contract  boiler  plate 
o  Specification 
o  Commander's  policies 

14.  Risk  Management 
Issue 

Standardize  the  methodology  and 
criteria  to  be  used  for  Risk 
Management. 

Recommendations 


o  Develope  a  handbook  of  Risk 
Management  methodologies  and 
criteria  to  be  used  for  procure¬ 
ment.  The  handbook  should  in¬ 
clude  several  Risk  Management 
approaches  so  that  the  program^ 
procurement  office  can  select  a 


specific  approach  for  a  given 
procurement. 

o  Validate  the  multi-attribute 
utility  technique. 

oBrief  SD  procurement  personnel. 

o  Establish  a  policy  on  RFP 
utilization. 
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Abstract 


In  1976  Hughes  Aircraft  Company's 
Space  and  Communications  Group  formed 
a  task  force  to  develop  specific  cost 
reduction  proposals  which  would  improve 
Company  productivity.  These  proposals 
defined  improved  engineering  manage¬ 
ment  techniques  which  would  achieve  a 
greater  reduction  in  design  deficiencies 
and  thus  improve  the  effectiveness  of  the 
technology.  These  techniques  are  dis¬ 
cussed  in  this  paper  and  include  1)  tech¬ 
nical  practices,  2)  design  checklists, 

3)  design  validation  reviews,  and  4) 
design  problem  summaries. 

Hughes'  engineering  management 
approach  establishes  design  authority  and 
responsibility.  Engineering  guidelines, 
which  include  past  design  deficiencies  and 
checklists  of  key  development  considera¬ 
tions,  are  maintained  at  the  department 
level.  This  is  an  important  aspect  of  mis¬ 
sion  assurance  to  ensure  that  mistakes 
are  not  repeated  and  important  considera¬ 
tions  are  not  missed.  Designs  are 
checked  by  knowledgeable  specialists. 

New  design  deficiencies  are  identified, 
guidelines  modified,  and  future  designs 
are  checked  for  these  deficiencies. 

These  techniques,  when  properly  applied, 
should  not  only  reduce  design  deficiencies, 
but  improve  the  effectiveness  of  the  tech¬ 
nology  and  provide  mission  assurance  at 
a  reasonable  cost. 


neering  productivity.  The  effectiveness 
of  these  techniques  or  tasks  are  a  key 
consideration  in  developing  a  cost  effec¬ 
tive  mission  assurance  plan.  Effective 
mission  assurance  tasks  are  those  which 
are  useful  in  tei  ms  of  .preventing  the 
occurrence  of  design  deficiencies  or  in 
detecting  them  early  in  the  development 
process.  A  cost  effective  assurance  plan 
considers  the  value  of  the  tasks  being 
performed. 


Engineering  deficiencies  cost  money. 
The  cost  of  most  deficiencies  is  a  function 
of  the  time  lag  between  the  mistake  and  its 
point  of  detection;  the  most  costly  mis¬ 
takes  can  be  those  which  remain  undetec¬ 
ted  until  operational  use.  This  class  of 
deficiencies  can  jeopardize  mission 
success. 


A  second  class  of  engineering  defi¬ 
ciencies  may  not  affect  mission  success, 
but  they  do  increase  engineering  cost  thus 
lowering  productivity.  These  are  detec¬ 
ted  prior  to  operational  use. 

The  following  illustrates  how  engi¬ 
neering  deficiencies  lower  productivity: 


Productivity 


Number  ol  units 


prodi 


/Indirect  \ 

/  Rework  \ 

and  direct! 

and  rede-\ 

labor  used!  + 

sign  labor 

to  produce! 

used  to  1 

items  / 

correct  / 

\  / 

\errors  / 

Introduction 

One  of  the  major  goals  of  Hughes 
Aircraft  Company  is  to  perform  success¬ 
fully  in  every  aspect  of  the  Company's 
business  activity.  At  Hughes  mission 
assurance  is  viewed  as  those  techniques 
which  enhance  program  success.  The 
Hughes  Space  and  Communications  Group 
(SCG)  Technology  Division  has  developed 
some  improved  techniques  for  minimizing 
design  deficiencies  and  improving  engi¬ 


It  is  apparent  from  the  above  that 
there  is  a  cost  benefit  to  do  business  in  a 
way  such  that  few  mistakes  are  created. 
Effective  technologies  are  those  which 
minimize  the  cost  of  engineering  mis¬ 
takes;  ineffective  technologies  create  too 
many  mistakes  and  take  too  long  to  cor¬ 
rect  them. 

Many  attempts  have  been  made  to 
reduce  substandard  products.  Some  prog- 


ress  has  been  made  in  manufacturing 
through  quality  controls,  quality  circles, 
and  sc reening  tests .  However,  attempts 
to  reduce  design  deficiencies  through  a 
process  of  design  requirements  including 
formal  design  reviews  is  not  always  effec¬ 
tive.  Design  reviews  may  not  consider 
design  problems  and  critical  design 
details  for  the  following  reasons: 

1)  Individuals  performing  reviews 
are  not  familiar  with  specific 
design  details;  consequently,  they 
have  not  been  able  to  identify 
costly  deficiencies. 

2)  Design  engineering  may  consider 
these  formal  reviews  as  customer 
relations  or,  in  the  case  of  com¬ 
pany  reviewers  (e.g.  ,  reliability 
engineers),  as  an  interference 
with  the  design  process. 

Design  engineering,  when  properly 
executed,  includes  the  total  product 
requirements  (i.e.,  conceptual  design  to 
field  use).  Product  deficiencies  indicate 
a  breakdown  in  this  process.  Effective 
engineering  discipline  at  the  working  level 
can  reduce  design  deficiency  costs. 

Hughes  has  made  an  attempt  to 
improve  the  effectiveness  of  the  technol¬ 
ogy  through  a  system  of  working  level 
checks  and  balances  administered  at  the 
engineering  department  level.  These 
techniques  are  discussed  in  the  para¬ 
graphs  which  follow. 

Cost  Effectiveness  Task  Force 

In  September  1976  Hughes  SCG  execu¬ 
tives  authorized  the  formation  of  a  task 
force  to  develop  specific  proposals  which 
would  lead  to  significant  reductions  in 
SCG  costs.  The  effort  was  chartered  for 
1  year  with  *-60  members,  a  manager,  and 
an  Executive  Review  Committee.  Produc¬ 
tivity  improvements  was  the  primary  focus 
of  the  task  force. 

One  aspect  of  the  task  force  activity 
was  a  study  of  the  institutional  deficiencies 
which  Contribute  to  design  and  manufactu¬ 
ring  errors.  It  was  concluded  that  these 
errors  result  in  significant  cost  elements. 
Error  prevention  and  early  detection  could 
reduce  error  costs  and  were  considered 


to  be  the  general  solution  to  this  problem. 
Identified  Institutional  Weaknesses 

The  task  force  performed  an  evalua¬ 
tion  of  the  causes  of  design  and  manufac¬ 
turing  errors.  It  was  concluded  that  the 
following  general  deficiencies  contributed 
to  design  error  costs: 

1)  Experience  gained  from  past 
design  problems  was  not  always 
incorporated  in  new  designs. 

2)  There  was  insufficient  checking 
of  designs  by  knowledgeable 
specialists . 

Some  design  problems  appear  to  be 
repetitive.  For  example,  design  errors 
were  not  always  remembered.  These 
errors  were  probably  remembered  by  the 
senior  staff  and  thus  avoided  in  their 
designs.  However,  there  was  no  effec¬ 
tive  means  for  transmitting  the  lessons 
learned  to  new  personnel. 

Design  reviews  tended  to  be  presenta¬ 
tions  of  the  design  to  the  customer  and 
were  not  sufficiently  effective  in  terms  of 
detecting  product  deficiencies.  The  indi¬ 
viduals  performing  the  review  were  not 
always  familiar  with  the  specific  design 
details;  consequently,  they  were  unable 
to  identify  and  correct  real  deficiencies. 

Another  identified  weakness  was  that 
there  did  not  appear  to  be  sufficient  feed¬ 
back  from  design  and  manufacturing 
errors.  An  individual  who  made  a  mis¬ 
take  and  did  not  know  it  could  conceivably 
continue  to  make  the  same  mistakes.  In 
addition,  controls  may  not  have  been 
established  to  make  sure  the  mistake  is 
not  repeated  by  others.  Perhaps  the 
weakness  stems  from  confusion  in  con¬ 
cepts  of  corrective  action.  For  exam¬ 
ple,  Quality  Control  considers  hardware 
rework  to  be  adequate  corrective  action, 
and  Reliability  Engineering  considers  a 
design  change  as  corrective  action.  How¬ 
ever,  from  the  error  prevention  stand¬ 
point,  corrective  action  is  complete  when, 
in  addition  to  the  above  corrections,  the 
individual  involved  in  the  mistake  under¬ 
stands  the  correct  approach  and  the  organ¬ 
ization  has  established  some  form  of  check 
to  ensure  that  this  error  cannot  be 
repeated . 
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Based  on  the  above  assessment,  the 
following  task  force  objectives  were 
identified: 

1)  Prevent  repetitive  technical 
problems . 

Z)  Identify  design  weaknesses 
early  in  development  cycle. 

3)  Feedback  design  errors  to 

individuals  who  made  the  mis¬ 
take  and  establish  controls  to 
ensure  errors  are  not  repeated. 

Task  Force  Recommendations 

During  the  period  that  the  task  force 
was  operative,  approaches  which  could 
reduce  design  deficiencies  were  exam¬ 
ined.  Specific  proposals  designed  to  cor¬ 
rect  Institutional  weaknesses  were  made 
to  the  Executive  Review  Committee.  The 
task  force  recommended  implementation 
of  the  following  tasks. 

1)  Technical  practices 

•  Maintain  description  of  past 
design  problems  and  the  cor¬ 
rect  procedure  to  be  followed 
to  prevent  these  problems  in 
future  designs . 

•  Ensure  new  employees  under¬ 
stand  past  problems  and  cor¬ 
rect  design  procedures  to  be 
followed. 

Z)  Design  checklists 

•  Maintain  list  of  key  develop¬ 
ment  considerations. 

•  Use  checklist  during  develop¬ 
ment  process  to  ensure 
important  development  steps 
are  not  missed. 

3)  Design  validation  reviews 

•  Develop  plan  for  checking 
designs  for  deficiencies, 
and  utilize  it  during  develop¬ 
ment  process. 

•  Reviews  are  to  be  performed 
by  senior  specialists. 


4)  Design  problem  summary 

•  Maintain  list  of  current 
design  problems. 

•  Identify  correct  design  prac¬ 
tice  to  be  followed  in  the 
future. 

•  Take  steps  to  prevent  recur¬ 
rence  of  design  problems 
including  issuing  new  techni¬ 
cal  practices,  modifying 
checklists,  and  checking  for 
deficiency  during  validation 
process . 

Improving  Engineering  Effectiveness 

Technology  Division  management 
decided  to  improve  the  effectiveness  of 
its  operation,  and  the  task  force  recom¬ 
mendations  were  to  be  implemented  at 
the  department  level. 

Each  department  had  its  ownapproach 
toward  reducing  design  errors.  Working 
level  design  reviews  were  not  always 
effective.  Lessons  learned  on  one  pro¬ 
gram  or  in  one  department  were  not  neces¬ 
sarily  transmitted  to  others.  In  some 
instances  there  was  insufficient  learning 
from  past  mistakes.  It  was  decided  to 
upgrade  the  design  error  prevention  and 
correction  capability  of  the  engineering 
departments . 

The  first  step  was  to  prepare  exam¬ 
ples  of  technical  practices  and  design 
checklists  along  with  sample  formats.  A 
draft  of  a  design  validation  plan  was  pre¬ 
pared.  One-on-one  meetings  were  held 
with  each  laboratory  manager  and  then 
with  the  department  managers;  the  need 
for  improvement  was  discussed  at  these 
meetings.  Each  department  manager  was 
requested  to  support  the  activities  and  to 
assign  an  individual  to  complete  the  tasks. 

The  next  step  was  to  meet  with  the 
individual  responsible  for  preparing  these 
guidelines  and  establish  start  and  com¬ 
pletion  schedules.  Table  1  shows  the 
product  types  and  disciplines  that  were 
involved  in  this  endeavor.  Although  these 
tasks  had  secondary  priority  to  the  nor¬ 
mal  work  load,  everyone  was  expected  to 
complete  the  tasks  in  a  reasonable  time 
period. 
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TABLE  1 .  TECHNOLOGY  DIVISION  PRODUCT  TYPES  AND  DISCIPLINES 


Communication  Laboratories 

RF  cables 
RF  switches 
Waveguide  switches 
Ferrite  devices 
Latching  relays 
Filters 
Antenna 
Mechanical 
Electrical 

Microwave  integrated  circuits 
Power  supplies 
Power  FET  amplifiers 
RF  electrical  design 
Digital  circuits 

Controls  and  Data  Systems  Laboratories 

Guidance  systems  design 
Electromechanical 


T&C  circuit  design 
T&C  test  equipment 
Digital  unit  design 
Microprocessor  flight  software 
Controls  and  ground  test  equipment 
Hardware 
Software 
Product  design 

Space  Vehicles  Laboratories 

Design  integration 
Thermophysics 
Stress 
Dynamics 
Propulsion 
Power  sources 
Batteries 
Solar  arrays 
Power  electronics 


The  first  task  was  the  preparation  of 
technical  practices.  Technical  practices 
are  the  design  error  memories  for  the 
departments.  They  were  generally  docu¬ 
mented  on  a  single  sheet  of  paper  and  in¬ 
cluded  the  following: 

•  Design  problem 

•  Design  solution 

•  Recommended  design  approaches 

Each  department  was  requested  to  docu¬ 
ment  its  past  problems  on  the  above 
format . 

Department  managers,  responsible 
for  their  particular  product  type  or  dis¬ 
cipline,  ensured  that  technical  practices 
were  maintained  and  properly  utilized. 
Past  problems  and  the  correct  approach 
were  to  be  explained  to  new  engineers  so 
that  problems  would  not  be  repeated. 
During  the  design  validation  process 
new  designs  were  to  be  checked  to  ensure 
they  were  free  of  these  deficiencies. 

New  technical  practices  were  to  be  issued 
as  new  design  deficiencies  were  encoun¬ 
tered  thus  continuing  the  learning  process. 

The  next  step  was  the  preparation  of 
design  checklists.  Each  department  was 


requested  to  prepare  checklists  of  the 
key  development  considerations  for  their 
particular  product  type  or  discipline.  A 
provocative  question  format  was  used. 

The  idea  was  to  jar  the  memory  of  the 
designer  to  make  sure  that  he  did  not 
miss  key  development  steps. 

Because  supervisors  were  often  too 
busy  to  spend  much  time  with  their  design 
ers,  the  checklist  would  also  serve  as  a 
guide  and  important  development  steps 
would  be  accomplished.  As  new  develop¬ 
ment  steps  are  identified,  checklists  are 
modified  accordingly. 

Working  level  design' reviews  were 
considered  very  important  techniques  for 
reducing  design  deficiencies  .  The 
reviews,  as  practiced,  were  not  con¬ 
sidered  to  be  as  effective  as  desired.  In 
some  instances,  important  engineering 
disciplines  did  not  participate  in  the 
review  process  thus  omitting  important 
design  checks . 

It  was  decided  to  stress  the  impor¬ 
tance  of  these  reviews  throughout  the 
organization  and  department  managers 
were  given  the  responsibility  to  perform 
effective  reviews.  Secondly,  it  was 
decided  that  specific  objectives  be  esta¬ 
blished  for  the  reviews.  The  objectives 
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FIGURE  1.  DESIGN  PROBLEM  SUMMARY  FORM 


of  one  of  the  departments  are  as  follows: 

1)  Greater  efficiency  and 
productivity 

2)  Reduction  of  design  errors 

3)  Early  detection  of  errors  or 
deficiencies 

4)  Establishment  of  specific  prac¬ 
tices  which  avoid  errors  that 
have  occurred  (standard  design 
procedures) 

5)  Reduction  of  cost  through 
improved  producibility 

6)  Analysis  of  the  actions  to  cope 
with  technical  problems 

7)  Optimal  application  of  resources 
to  produce  error -free  designs 

8)  Design,  integration,  and  testing 
commonality 

The  name  "design  validation"  was 
selected  so  that  these  working  level  re¬ 
views  would  not  be  confused  with  customer 
reviews.  The  objective  of  the  design  vali¬ 
dation  review  was  to  confirm  the  correct¬ 
ness  of  the  design  by  checking  it  for  defi¬ 
ciencies.  Each  department  prepared  a 
design  validation  plan  and  then  performed 
its  validation  reviews  in  accordance  with 
this  plan. 

The  remaining  technique  to  be  imple¬ 
mented  was  a  system  where  new  design 
deficiencies  would  be  identified  and  cor¬ 
rected  and  the  design  guidelines  appro¬ 
priately  modified.  As  previously  men¬ 
tioned,  it  was  important  that  the  mistakes 
be  fed  back  to  the  responsible  individual 


so  that  he  could  learn  the  correct  pro¬ 
cedure  and  that  new  checks  be  established 
so  that  the  repetition  of  errors  could  be 
prevented. 

The  Design  Problem  Summary  Report 
was  selected  for  this  purpose.  This 
report  summarizes  the  design  problems 
for  management  review.  Each  depart¬ 
ment  is  requested  to  document  the  steps  it 
had  taken  to  ensure  that  the  design  defi¬ 
ciency  is  not  repeated.  The  form  shown 
in  Figure  1  is  used  to  document  this  pro¬ 
cess  and  thus  show  how  institutional  cor¬ 
rective  action  is  achieved. 

An  attempt  has  been  made  to  develop 
simple  and  practical  approaches  to  reduce 
design  deficiencies .  Documentation  is 
held  to  a  minimum  so  that  cost  effective¬ 
ness  can  be  achieved.  It  is  believed  that 
the  above  techniques  offer  a  practical 
approach  to  achieve  a  reduction  in  design 
deficiencies. 


Cone  lusion 

A  primary  objective  of  Hughes  is  to 
perform  successfully  in  every  aspect  of 
our  business  activities.  To  achieve  this, 
designs  must  have  minimal  deficiencies 
and  those  remaining  deficiencies  must  be 
corrected  early  in  the  development  pro¬ 
cess.  Furthermore,  we  must  operate 
effectively  to  ensure  that  our  programs 
remain  within  budgets . 

The  Technology  Division  believes  and 
the  experience  to  date  suggests  that  the 
most  cost  effective  way  to  reduce  design 
deficiencies  is  to  make  improvements  at 
the  working  level.  More  effective  working 
level  design  reviews  performed  by  knowl¬ 
edgeable  specialists  who  are  familiar  with 
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TABLE  2.  PRODUCT  DEFICIENCY  COST  REDUCTION  -  OUESTIONS  AND  ANSWERS 


QUESTIONS 

ANSWERS 

1 

Are  your  error  costs  high? 

1.  If  answer  is  yes 

•  Redesign  and  rework  costs 

•  Need  to  make  institution  improvements 

•  Warranty  costs 

•  Product  liability  costs 

which  will  reduce  these  costs 

2. 

Have  you  taken  steps  to  reduce  error  costs? 

2.  If  answer  is  no 

•  Corporate  memory  of  past  design  errors 

•  Need  to  document  past  design  errors  and  the 
correct  design  solutions 

•  Checklist  of  key  design  elements 

•  Need  to  develop  design  checklists  so  impor¬ 
tant  design  considerations  are  not  missed 

•  Effective  design  review  discipline 

•  Need  to  validate  designs  to  ensure  that: 

—  Design  free  of  past  errors 

—  New  designs  are  free  of  past  errors 

—  Key  design  elements  considered 

—  Checklist  items  are  accomplished 

—  Experienced  personnel  evaluated 
design 

—  Effective  design  approach  is  utilized 

3. 

Are  manufacturing  personnel  reducing 
defects? 

3.  If  answer  is  no 

•  Do  you  have  quality  circles? 

•  Involve  workers  in  solution  of  their  defect 
problems 

4. 

Do  tests  and  inspections  identify  design  and 
manufacturing  weaknesses? 

4.  If  answer  is  no 

•  Qualification  test 

•  Change  tests  and  inspections  to  improve 

•  Inspection 

•  Manufacturing  test 

effectiveness 

5. 

Can  you  identify  product  deficiencies  from 
your  information  system? 

5.  If  answer  is  no 

•  Design  errors 

•  Improve  information  system  so  that 

•  Manufacturing  errors 

deficiencies  can  be  identified  in  a 

•  Ineffective  tests  and  inspections 

timely  manner 

6. 

Do  you  have  an  effective  system  for  pre¬ 
venting  deficiencies? 

6.  If  answer  is  no 

•  Design  deficiencies 

•  Manufacturing  deficiencies 

•  Test  and  inspection  deficiencies 

•  Institutional  deficiencies 

•  Need  effective  corrective  action  system 

design  details  should  achieve  both  a  better 
error  prevention  and  early  detection  capa¬ 
bility.  Hughes  believes  that  the  mission 
assurance  objectives  can  be  best  achieved 
through  the  use  of  these  engineering  man-* 
agement  disciplines  properly  applied  at  the 
working  level. 


Recommendations 

It  is  recomi^iended  that  the  government 
encourage  contractors  to  assess  the  effec¬ 
tiveness  of  their  technologies,  identify 
weaknesses,  and  make  improvements. 
Table  2  can  be  helpful  in  this  process. 
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OVERVIEW 

•  MIL-STD-1521A 

•  Perspective 

•  Reviews  and  Audits 

•  More  Reviews  and  Audits 

•  Still  More 

•  Cognizant  Engineers 

•  Technology  Previews 

•  Periodic  Briefing  and  Reviews 

•  Contractor/Customer  Emphasis 

•  Common  Contractor  Faults 

•  Common  Customer  Faults 

•  Recommendations 

MIL-STD-1521A  REVIEWS 


Medium  Complexity 
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MORE  REVIEWS/AUDITS 


•  System  Effectiveness  Audits 

-  Reliability 

—  Maintainability 
—  Human  Engineering 
—  Corrosion  Controi 

-  Etc 

•  Parts  —  Materials  —  Processes  Control  Board  (PMPCB) 

•  Range  Safety  Reviews 

•  Explosives  Hazards  Review 

•  Telemetry  Frequency  Allocation  Review 

•  Safety  Reviews 

•  Quality  Audits 

•  Award  Fee  Evaluations 


STILL  MORE  REVIEWS 

•  Independent  Readiness  Reviews 
—  Customer  Heavyweights 
—  Considerable  Depth  —  Good  Audit  Procedure 


HARDWARE  DESCRIPTIONS 

•  Similar  to  Part  Number  _ 

•  How  Many  Vehicle _ 

GSE _ 

Spares _ 

•  Estimated  Cost 

•  Availability 

•  Number  and  Complexity  of  Drawings 

•  Make/Buy  Recommendation 

•  Next  Assembly 

•  Sketches 


•  Vehicle  Flight  Readiness  Review 
—  Moderate  Preparation 
—  Formal  Presentations 
—  Executive  Overview 


COGNIZANT  ENGINEERS* 

Chief  Program  Engineer 


•  ELECTRONICS 

-  Telemetry 

-  Elec/Electronics 

•  MECHANICAL 

~  Ordnance 

-  Separation 

•  STRUCTURAL 

-  Design 

-  Strength 

-  Weighte 

-  VibraOon,  Shock,  Acoustics 

•  PROPULSION 

-  Motor 

-  SpInSyelem 


•  SYSTEM  PERFORMANCE 

'  Trajectory 
"  Stabllity/Control 

•  EFFECTIVENESS 

-  Reliability 

'  Malnteinability 
Safety 

•  TEST 

Oeveiopment 

-  Quelificctlon 

-  Acceptance 
In-Houee 

-  Vendor 


TECHNOLOGY  PREVIEWS 


•  Cognizant  Engineer  Reviews  System 

•  Senior  Staff  and  Designers  Critique 

•  Design  Review  Agreements  (DRA’s)  Result 

—  Originated  by  Prime  Technology 
—  Signoffs  by  Interfacing  Technologies 

•  DRA's  Widely  Distributed 

—  Quality 
—  Manufacturing 
—  Tooling 
—  Procurement 
—  Design  Technologies 
—  Test  Laboratories 
—  Field  Station 


'Stiown  tor  •  Pwtieulir  Proomn  -  Ollwra  Will  OHftr.  Larger  Program  Cognizant 
Enginaart  An  UtmUy  Product  Orlantad. 


CvAlwalion 


PERIODIC  BRIEFINGS  AND  REVIEWS 


COMMON  CUSTOMER  FAULTS 


•  TlirW'TImM  W— My  Englw— ring  Ravlaw 

-  Informal  Round-tabI#  Stylo 

-  Program  Enginaar  Modarataa 

-  Cognizant  Enginaora  Oiacuaa 

-  Early  Surfacing  of  ProMoma 

-  Diract  Aaatgnmant  of  Taaka/Raaponaibilitiaa 

-  1/2>Hour  Maximum 

•  Weakly  Program  Review 

-  Viawgrapti  Preaentationa 

-  Program  Manager 

-  Department  Heada  (Engineering,  Manufacturing, 
PrMuremont,  Quality,  Fiacal,  Etc.)  DIacuaa 

-  Direct  Feedback  on  Problem  Araaa 

-  1-2  Houra  Maximum 

•  Monthly  Executive  Reviawa  (30-30) 

-  Senior  Executive  Praaant  Moderatea 

-  Program  Managera  DIacuaa 

-  1 /2-Hour  Maximum  Per  Program 


"‘/A 

I  /  \  /  V  Customsr  Emphasis 


COMMON  CONTRACTOR  FAULTS 


•  Lack  of  Candidneaa 

•  Evasiveneaa 

•  Lack  of  Preparation 

•  Cuatomera  Competence  Undereatimated 

•  Lack  of  Money  (Did  Not  Bid  Enough  to  Cover  Reviewa,  and 
Certainly  Not  Enough  to  Cover  Action  Itema) 

•  Too: 

—  Elaborate  —  Long 

—  Boring  —  Short 

•  Not  Enough  Subatance 

•  Not  Well  Organized 

•  No  Effective  Exchange  With  Cuatomer's  Technical  People 


a  Often  Inflexible 

•  Usually  Sends  Too  Many  Attendees 

•  Action  Items  Often  Trivial.  Many  Questions  Adequately 
Answered  During  Review  End  Up  as  "Action  Items" 

•  Some  Attendees  Appear  to  Want  (or  Need)  Tutoring 

•  Technicai  Emphasis  Not  Weli  Balanced 

•  Frequently  Wants  More  Than  He  Paid  For 

•  Frequently  Late 

—  For  Review 

—  With  Comments 

—  With  Reaction  to  Action  Items 


RECOMMENDATIONS 


•  Involve  Customer  Technical  Experla  In  Early  Contractor 
Review  Processes 

•  Place  More  Emphasis  on  Technical  Interchange  (T.l.) 
Meetings  -  Less  On  Large-Scale  Reviewa 

•  Revise  MIL-STO-1521A  To  Include  Tl'a  and  Working  Groups 


example 


DESIGN  REVIEW  “ESCAPES'' 

An  examination  of  some  "escapes"  from  the 
design  review  process,  in  the  hope  of 
sealing  off  a  few  of  the  cracks  that 
things  have  been  known  to  fall  through. 

E.H.  (Gene)  Barnett 

Manager,  Space  Systems  Product  Assurance 
Reliability  and  System  Satety  Department 
TRW  -  Defense  and  Space  Systems  Group 
Redondo  Beach,  Ca. 

A  review  of  the  kind,  and  causes,  of  expen¬ 
sive  escapes  from  our  design  review  process 
has  led  me  to  an  awareness  about  at  least 
one  shortcoming  of  the  process,  one  which 
I  think  deserves  some  attention. 

In  the  few  minutes  I  have  for  this  presen¬ 
tation,  a  few  case  histories  will  help  set 
the  stage.  Then  I'd  like  to  discuss  ways 
to  prevent  these  escapes  with  you. 

THE  PROBLEM: 

During  test,  short  circuits  developed 
between  copper  circuit  traces  on,  and 
mounting  bolts  through,  a  multi-layer  PC 
board. 

THE  CAUSE: 

The  board  layout  did  not  leave  enough 
clearance  between  traces  and  mounting 
bolts. 

WHAT  WENT  WRONG; 

These  boards  were  reviewed  by  reliability 
engineers  to  study  the  physical  relation¬ 
ships  between  circuit  traces  and  piece 
parts.  In  order  to  clearly  see  the  traces 
in  the  board,  transparencies  of  the  copper 
artwork  were  layed  one  atop  the  other  and 
viewed  through  the  light. 

But  the  transparencies  showed  only  the 
copper  artwork,  not  the  position  of  holes 
for  the  mounting  bolts. 

example 


9  PROBLEM:  arclni. 


9  CAUSE:  In8dc<iu4t«  tpaelng  b«(w<on  alttaanct.  During  poctlng,  cht  pace's 

tarblnal  board  was  not  bald  (Ira,  buc  was  allowsd  ca  find  ica 
own  poslcion  as  dacaralnod  by  acrasaas  an  connaecing  viraa.  Thia 
rssulttd  in  random  spacing  baewtsa  coapooancs. 


I  FIX:  Sea#  earslnal  boards  w«-ra  affacead;  oehara  wars  net.  lapaaaiva 

toseing  and  raword  ware  naadod  to  find  affaccad  parts,  unpoc  thas, 
capuir  with  a  naw  holding  lixcurs,  and  cacasc  tha  uaica. 


i  COW^ENTS :  All  eolaranca  and  aanufactuting  flxturaa  should  ba  ravlawad  by 

Quality  aed  Manufacturing  at  unit  lava!  daalga  ravlawa. 


9  PROBLEM:  lapropar  funccioolog  of  an  ap-aap  in  a  voltaga  ragulacar  circuit. 


#  CAUSE:  The  part  was  als-appllad,  under  a  coebinatlan  of  cendltiona  —  high 

teaparatura,  low  input  voltaga  and  Output  charactarlatics  of  tbs  op>aap. 


9  FIX:  Ba-seracn  all  op-oapt  and  selactlvely  pick  thosa  with  spaclal  par~ 

foruonca  characccrlslcics.  Xeplaca  all  op-asps  in  (light  units 
with  spacially  scraanod  davicoa,  and  rscaat. 


9  COM.MENTS;  A  sore  thorough  review  of  worst  casa  aasuspeiona  by  rallabllity,  and 
of  part  applications  by  Cospenancs  and  daslgn  anglnaars,  aay  bavt 
uncovarad  this  problas  at  cha  Intaraadiata  Dealgn  Kaviaw. 


EXAMPLE 


9  problem:  Unabla  ta  aalntaln  raguirad  gyro  teaparatura. 


9  CAUSE:  daat  loss  greater  than  asauaad  In  charoal  aaalysia  of  haac  losa  paths. 


•  FIX:  kadaslgn  and  repair  all  flight  units. 


A  COMMENTS:  A  oora  cii:.''-'.  review  of  cha  charoal  design  assuaptlona  would  hava 

uncovarad  this  problua  a..  zi.t  Critical  Daalga  Raviaw. 


EXAMPLE 


9  PROBLEM:  Switch  would  not  accuata  properly. 


b  CAUSE:  wtru»  broke  is  vibration  taec.  lUauUad  froa  poor  product  anglaaarlag. 

lnauitiel«nt  atrain-ralltf  on  cenaactor  wiraa. 

•  FIX:  bodasiga  and  rework  alvoad-bullt/taatad  unite. 


•  COMMENTS:  An  a»»uably  drawing  review  by  ralUbliity  aaglsaars  would  Uva  praventad 

thia  problaa. 


example 


•  PROBLEM:  Cracks  and  dalaslnation  cccurrad  In  a  aelar  panel  atruccura,  (bllawlsg 

cold-soaking  of  tha  panel. 

f  CAUSE:  irronaoua  aaauaptions  (regarding  thamal  aepanslon  tnefftetanta)  lad  to 

undar-asclaacing  Cha  degrta  of  flexing  chat  would  occur  with  thermal  cycling. 

C  COST;  Mdti.^n.  rabulld,  retest,  schedule  slip. 

•  COMMENTS:  A  more  thorough  review  of  nodeling  assumption  ta  a  cheraal  subayetaa  or 

a  solar  array  review  should  hava  uncovarad  this  problca  before  building 
cha  aolar  array. 

I'm  sure  we  all  have  an  assortment  of  war 
stories  to  tell.  And  no  wonder;  think  of 
the  incredibly  complex  equipment  we  build. 
In  every  one  of  our  designs,  there  are  a 
million  nooks  and  crannies  where  gremlins 
can  hide  out,  lieing  there  like  so  many 
spring-loaded  booby  traps  wanting  to  go 
off  in  our  faces  (just  when  we  think  we've 
made  it  safely  through  the  pass!) 

Behind  each  analysis  and  every  decision  we 
make,  there  are  defects  in  our  design 
process  itself:  Simplifying  assumptions; 
erroneous,  inaccurate  and  uncertain  data; 
insufficient  attention  to  details;  lack  of 
historical  perspective  by  new  people;  just 
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plain  goofs;  and  cracks  for  things  to  fall 
through. 

After  we've  eradicated  a  million  gremlins, 
the  million-and-one-th  gremlin  is  still 
lurking  in  the  shadows,  waiting  to  ambush 
us. 

Time  is  the  real  culprit.  We've  got  dead¬ 
lines  to  meet,  decisions  to  make,  schedules 
to  hold  and  budgets  to  save.  We  do  our 
best,  but  we're  only  human. 


Sure  enough,  all  the  appropriate  words  get 
said  in  all  the  appropriate  sections  of 
MIL-STD-1521 ,  and  in  every  reliability 
and  systems  engineering  program  plan  ever 
written  -  Thou  shalt  review  assumptions, 
data  sources,  analytical  methods,  etc. 

And  indeed,  we  try  to.  But  again.  Time 
is  the  culprit. 

A  typical  design  review  agenda  lays  aside 
a  (relatively)  few  minutes  to  dozens  of 
topics  deemed  important  to  the  hardware 
being  reviewed  that  day  -  electrical  pro¬ 
perties,  specifications,  EMC,  vulnerability, 
reliability,  environment,  interfaces, 
power  consumption,  testability,  manufac¬ 
turability,  you  name  it. 

The  room  fills  with  engineers,  managers, 
support  troops,  customers,  on-lookers, 
advisors  and  critics.  A  Responsibile 
Engineer  gives  an  explanation  (and  some 
times  a  defense)  of  the  results  of  his 
analyses,  a  few  probing  questions  are 
asked,  an  action  item  or  two  is  taken,  and 
the  next  presenter  comes  to  the  podium. 

If  a  presenter  takes  too  long  -  if  too  many 
probing  questions  are  asked,  or  too  many 
axes  get  ground  -  the  agenda  falls  behind, 
the  audience  gets  fidgety  and  five  o'clock 
draws  near. 

The  problem,  I  think,  is  that  we  don't  give 
enough  time  to  reviewing  the  integrity  of 


the  process  the  design  went  through  along 
the  way  to  the  Design  Review  itself. 

MHY  DESIGN  REVIEWS  HISS  THESE  THINGS  ... 

•  DESIGN  REVIEWS  TEND  TO  FOCUS  THENSELVES  ON  THE  t)£SICK  ... 

#  Not  oa  th«  loading  up  tc  tht  dsaign  . . . 

#  1b  spit*  of  what  It  Id  ^rofraa  flana  and  Nil  Staadardt  ... 

•  Thara  Iwa!  lan't  anoufh  tlaa  that  cao  ba  allottad  for  aach 
agblact  Id  a  daalsn  ravlaw. 

•  SOi  these  THINGS  TEND  TO  GET  NEGLECTED  ... 

%  A  crltlrat  and  thorough  aaaalnatLon  of  all  aaaoaptlooa  ... 
d  T>ia  ()uaUt>  of  tha  undarlylng  data  tcaalf  .  . 

•  AAalytiral  coola  and  trchniquaa  Jtad 

^  Nlaatng  data,  alaplt f icat Iona  and  approalaal Iona  ... 

%  Up-to-data-naaa  of  dtawinga,  ate.,  fro*  tdttch  work  la  dona  ... 

%  tapariw'.ca  and  qualKlcat tona  [by  naaaj  of  Individual#  doing  tha  work. 


For  lack  of  a  better  phrase.  I've  coined 
my  proposal  ASSUMPTIONS  REVIEWS,  though 
the  title  is  a  little  misleading  and  con¬ 
fining. 

I  propose  that  all  major  design  reviews  be 
preceeded  by  a  series  of  ' *  'mptions 
Reviews  which  are  exempt  m  passing 
judgement  on  the  design  itself.  Rather, 
these  review  should  be  charged  with  examin¬ 
ing  the  assumptions,  techniques  and  qualif¬ 
ications  that  underly  the  results  will  be 
presented  at  later  design  reviews. 

I  propose  that  these  Assumptions  Reviews  be 
formatted  in  the  same  way  as  normal  design 
reviews  -  complete  with  a  committee,  a 
chairman,  data  packages  (describing  assump¬ 
tions,  etc.),  action  items  and  formal 
closeout.  Such  a  review's  sole  charter  is 
to  assure  the  later  Design  Review's  chair¬ 
man  that  the  underpinnings  and  thoroughness 
of  the  analyses  leading  to  the  presented 
design  are  sound  and  trustworthy. 

I  believe  that  each  of  several  disciplines 
ought  to  have  dedicated  Assumptions  Reviews 
-  for  example,  reliability,  thermal, 
vibration,  etc.  -  so  that  time,  attention 
and  talent  is  not  diluted  from  the  review 
at  hand,  and  that  each  review  ought  to  be 
across-the-entire  discipline  as  it  applies 
to  the  project  (ie.,  not  focused  on,  say, 
the  reliability  analyses  of  just  one  sub¬ 
system.  ) 


A  PROPOSAL  FOR  'ASSUHPTIORS  REVIEWS* 


•  REQOIRE  FWHPO.  PRE-RlViEVS  ... 


Structurtd  Ilk*  P«*I»S“. 

•cclOB  ICMA,  el«a*auc,  ate. 


iCtMA, 


•  FocuA  on  chA  inugrity  »t  thA  Aulrtl««l  i*r<»eAAA«A  thA  dAAita  wabi 
through. 


A  FOCUA  OB  thOAA  lAAUAA  OB  tho  ptACAAdlBg  aIIAA.  OOl  OB  tht  doAlgB  ItAOlf. 

Thu  it  BOt  A  dry  run  o(  th«  doalgn  rovtaw  co^ng  up  lACor. 


•  SopatAtA  rtvlAWA  for  aopArACo  Iaauaa  -  oba  for  rAllAblllty  AAAuaptloAA, 

ABothtr  for  chorMl  AAAuaptlooA,  a  third  oba  for  vlbratlaii  AaaunptlooA.  ate- 


•  THEN  AT  THE  FORMAL  DESIGN  REVIEWS  HE  NOW  NORMALLY  MOLD  ... 

a  ChalrMB  of  tha  Aaaunptlona  tavlavA  ahoild  raport  to  Cha  caa«lctaa  on 
hla  flndinga  In  that  ravlaw. 


Aaaura  cha  — r1~*i^  that  tha  undarirtng  proeaaAaa  tha  daalgs  want  through 
ara  truatworthy. 


DESIGNING  FOR  MISSION  ASSURANCE 


DESIGN  WORKSHOP 

MANNED  MANEUVERING  UNIT 
SPACE  OPERATIONS  SIMULATION 
DESIGN  REVIEW  PROCESS 

Leroy  J.  Duchartne 
Section  Manager 
Systems  Integration 


MISSION  ASSURANCE  -  DESIGN  WORKSHOP 


Designing  For  The  Shuttle  Era 

The  Design  Review  Process 

System  Design  Process: 

Hardware/Softwve/Mjn-Machine  Interface 

•  Conceptual  Phase  *  Functional  Baseline 

-  System  Requirements  Review 

•  Definition  Phase 

Allocated  Baseline 

*  System  Design  Review 

•  Development  Phase  - 

Preliminary  Design 

-  Preliminary  Design  Review 

- 

Detail  Design 

■  Critical  Design  Review 

- 

Build,  Code,  Test 

*  Test  Readiness  Review 

- 

System  Verification 

*  Configuration  Audits 

•  Integration  Phase 

System  Validation 

-  Preliminary  Ouatirication  Test 

System  Certification 

->  Final  Oualification  Test 

-  Certification  Board  Approval 

•  Oqerjlions  Phase  • 

Change  Orders 

•  Configuration  Review 

THE  DESIGN  REVIEW  PROCESS 

Military  Standard  1521A,  Technical  Reviews 
and  Audits  for  Systems,  Equipments,  and 
Computer  Programs  identifies  the  design 
reviews  to  be  conducted  during  the  life 
cycle  of  a  program.  In  addition  to  identi¬ 
fying  the  design  review  process,  it  is 
important  to  define  the  phase  of  the  pro¬ 
gram  in  which  the  review/audit  is  conducted 
and  the  objective  achieved  in  support  of 
the  hardware/software/man-machine  interface. 

The  Manned  Maneuvering  Unit  Program,  Space 
Operations  Simulation  (SOS)  and  Astronaut 
training,  was  selected  to  demonstrate  the 
application  of  MIL-STD-1521A  in  a  labora¬ 
tory  environment.  The  contract  with  NASA 
requires  that  the  SOS  facility  be  demon¬ 
strated  and  certified  prior  to  astronaut 
training. 


MISSION  ASSURANCE  -  DESIGN  WORKSHOP 


Tht  Design  Revian  Process 

Manned  Maneuvering  Unit  Program 

•  Space  Operations  Simuiation  And  Training 

•  Objectives:  Design  Verification  And  Astronaut  Training 

Conceptuai  Phase  -  Functionai  Baseiine  -  SRR  -  System  Requirements  Review 

•  SOW  Requirements 

High  Fidelity  Simulation  -  6  Degrees  Of  Freedom  -  Safely  Certified 
Shirtsleeve  And  Suited  Operation 
Credible  Malfunction  CapAility 

Simulation  Data  To  Be  Record^  Communications  And  Videol 
Recoveiy  To  Limited  Capability  In  3  Days 

•  Derived  Requirements 

Performance  IMMU  Mission  -  Tile  Inspection  And  Repair! 
Structures  (Stress  And  loads  AnalysIsl 
Safely  (Mockups,  Design,  Operations! 

Overall  System  -  HW/SW/MMI 

•  SRR  -  System  Specification  l"A"  Level  Specification! 

Trade  Study  Reports.  Functional  Flows, 

Operations  Concept 


CONCEPTUAL  PHASE 

The  Manned  Maneuvering  Unit  program.  Space 
Operations  Simulation  and  Training  portion 
of  the  contract  has  two  primary  objectives: 
design  verification  and  astronaut  training. 

The  simulation. requires  six  degrees  of  free 
dom  to  provide  astronaut  training,  must 
be  capable  of  supporting  both  shirtsleeve 
and  suited  operation,  and  must  provide  the 
capability  to  impose  credible  malfunctions 
to  ensure  proficiency  training.  The  simu¬ 
lation  facility  must  be  safety  certified, 
have  the  capability  to  record  all  video 
and  communications  during  the  training 
process,  and  be  designed  to  recover  from 
a  malfunction  in  three  days. 

Derived  requirements  were  identified  in 
support  of  performance,  structures,  safety 
and  overall  system  performance.  Safety 
considerations  were  applicable  to  mockups, 
design  criteria  and  MMU  operations.  The 
System  Requirements  Review  provides  an 
opportunity  to  present  the  results  of 
trade  studies,  functional  flows  and  the 
proposed  operations  concept.  User  con¬ 
currence  is  vital  to  the  success  of  the 
program  and  is  documented  in  an  "A"  level 
specification. 


MISSION  ASSURANCE  -  DESIGN  WORKSHOP 

Th»  Dgsign  Proctss 

Definition  Phase  •  Alloceted  Baseline  *  SI 

•  HardMare 

6  Degree  Of  Freedom  Carriage 
Mockups 

Manned  Maneuvering  Unit 
Test  ConductorffTun  Coordinator 
Console  (TC/RC) 

Space  Operations  Simulation  Console 
Communications  And  Video  System 
Safety  Equipment 

•  Man -Machine  Interface 

Astronaut  -  MMU 

Test  Conductor  And  Run  Coordinator 
SOS  Console  Operator 
Safety  Monitor 

•  SDR 

Simulation  Implementation  Plan  (Preliminary  System  Description  And  Allocation  Ooc.l 
Preliminary  Configuration  Item  -  Development  Specification  (Bl) 

Software  Requirements  Document  (B5  SpeciOcation) 


DEFINITION  PHASE 

The  definition  phase  results  in  an  allo¬ 
cation  of  requirements  to  hardware,  soft¬ 
ware,  and  the  man-machine  interface  and 
is  presented  at  the  System  Design  Review 
(SDR). 

The  SDR  presents  an  opportunity  to  define 
the  proposed  allocation  of  requirements 
and  is  documented  in  "B"  level  specifi¬ 
cations:  Bl  specification  for  hardware 
configuration  items  and  a  B5  specification 
for  software  requirements.  The  MMU  Space 
Operations  Simulation  also  required  a 
Simulation  Implementation  Plan  to  define 
the  Man-Machine  interface  and  program 
milestones. 

Definition  of  the  Test  Conductor,  Run 
Coordinator,  SOS  Console  Operator,  and  the 
Safety  Monitor  roles  and  responsibilities 
were  presented  at  the  SDR  to  obtain  user 
concurrence. 


MISSION  ASSURANCE  -  DESIGN  WORKSHOP 

The  Design  Review  Process 
Development  Phase  •  PDR  And  COR 

•  Hardware 

Subsystem  Analyses  And  Trade  Study  Results 
Performance  Data 

Math  Models  (Computer  Interface  Electronics) 

Drawings  And  Procurement  Specifications  (Cl-Buikt  To) 

Test  Specification 

•  Software 

Subsystem  Analyses  And  Trade  Study  Results 

Functional  Diagrams  And  Functional  Decomposition 

Data  Base  Analysis 

Software  System  Partitioning 

Computer  Program  Product  Specification  (CS<ode  To) 

Test  Plans  And  Procedures 

•  Man-Machine 

Personnel  Subsystem  Analysis 
Human  Factors  Analysis 
Certification  Plan 
Training  Plan 

Safety  Analysis  Reviav  And  Hazard  Analysis  Sheets 


-  System  Design  Review 
•  Software 

MMU  Performance  Dfuations 
Orbiter  Dynamics 

Contingency  (Malfunction  Procedures) 
Display  Formats  To  TC/RC 
Safety  Parameters 


DEVECOPMENT  PHASE 

The  development  phase  consists  of  two  , 
design  reviews,  preliminary  and  critical, 
a  test  readiness  review  and  configuration 
audits.  These  major  milestones  cover  hard¬ 
ware,  software  and  man-machine  interfaces. 

Hardware  elements  are  presented  via  sub¬ 
system  analyses,  trade  study  results, 
performance  data,  math  models  and  are 
documented  in  Cl  specifications,  covering 
both  build  to  and  test  procedures.  Soft¬ 
ware  elements  are  presented  via  functional 
diagrams  and  decomposition,  data  base 
analysis,  system  partitioning  and  are 
documented  in  C5  specifications,  covering 
both  code  to  and  test  procedures. 

The  man-machine  considerations  are  pre¬ 
sented  via  personnel  subsystem  analysis, 
human  factors  analysis  and  are  documented 
in  certification  and  training  plans, 
including  a  safety  assessment  via  hazard 
analysis  sheets. 


MISSION  ASSURANCE -DESIGN  WORKSHOP 

The  Design  Review  Process 
Development  Phase 

•  8uild-Code-Test  -  TRR  -  Test  Readiness  Review 

Hardware:  Fabrication  -  Subassembly  Tests  - 

Oualification  -  Subsystem  Intep.  Test 

Software:  Code  -  Unit  Test  -  Module  Test  - 
Subsystem  Intej.  Test 

•  System  Verification  -  FCA  -  Final  Confipuration  Audit 
HW/SW/MMI:  Physical  And  Functional  Audit  -  Against  "A"  Spec. 

Verification  Matrix  -  Analysis.  Test,  Inspection 
Requirement  Verification  -  Bl/BS  vs.  CliCS 
Throuqh  Put  Verification  -  Commands  And  Response 
Posilional/System  Training  -  TC/RC/SOS/Salety 
Evaluate  Reports  -  Structures,  Stress,  Safety 
Prxedures  IValldation  And  Operational!  -  Review 


DEVELOPMENT  PHASE  -  Continued 

The  test  readiness  review  is  conducted  for 
both  hardware  and  software  elements.  The 
qualification  of  hardware  and  the  module 
testing  of  software  are  reviewed  in  pre¬ 
paration  for  integrated  testing  at  the 
subsystem  level. 

A  final  configuration  audit  is  performed, 
both  a  physical  and  functional  audit,  to 
verify  design  conformance  against  the  "A" 
specification;  the  verification  may  be 
performed  by  analysis,  test,  or  inspection. 

The  MMU  SOS  system  verification  will  con¬ 
sist  of  through  put,  commands  and  response. 


training  and  certification  of  all  console 
positions,  a  safety  audit  and  a  review  of 
validation  and  operational  procedures. 


MISSION  ASSURANCE  -  DESIGN  WORKSHOP 

The  Design  Review  Process 

Integration  Phase  -  System  Validation  -  FQT  >  Final  Qualification  Test 
Mission  Simulation 

•  Unmanned  Test  And  Checkout  *  PQT 

•  Manned  Test  And  Checkout  (Shirtsleeve  And  Suited)  *  FQT 

•  Certification  Board  Approval 

•  Customer  Demonstration  And  Acceptance 

Operations  Phase 

•  Training  -  Normal  And  Contingency 

•  Change  Orders  -  Added  Capabilities  And  Enhancements 

•  Discrepancy  Reports 

Class  I  *  Certification  Board  Approval 

Class  II  -  Safety  Approval 

Class  III-  Test  Conductor  Approval 

•  Configuration  Management  -  Change  Board  Activities 

Technical  And  Programmatic  Assessment 


INTEGRATION  PHASE 

The  Integration  phase  consists  of  system 
validation  and  system  certification.  The 
MMU  preliminary  qualification  test  will 
consist  of  an  unmanned  test  and  checkout 
of  the  six  degree  of  simulation  hardware 
and  the  software  programs.  Certification 
board  approval  will  be  granted  after 
unmanned  and  manned  demonstration  against 
approved  procedures.  Customer  accep¬ 
tance  will  be  granted  after  a  demonstration 
of  the  total  mission  simulation,  including 
malfunction  and  recovery  capabilities. 

OPERATIONS  PHASE 

Astronaut  participation  will  consist  of 
f^U  proficiency  training  to  perform  a  tile 
inspection  and  tile  repair  mission,  includ¬ 
ing  normal  and  contingency  procedures. 
Change  orders  will  be  processed  to  add  ca¬ 
pabilities  and  enhancements;  discrepancy 
reports  will  be  utilized  to  perform  trou¬ 
bleshooting  and  repair.  Three  classes  of 
changes  have  been  defined  to  allow  proper 
approval  prior  to  continued  operations. 


MISSION  ASSURANCE  -  DESIGN  WORKSHOP 


Key  Issues 

•  System  Requirements  Review 
User  Concurrence 
m  Preliminary  Design  Review 

Design  To  Cost/Ufe  Cycle  Cost  Consideration 
«  Configuration  Audits 

Firm  Baseline  -  Documented 

Recommendation 

The  Design  Review  Process  Should  Stress  The  Role 
Of  Systems  Engiheering  And  User  Concurrence 
At  All  Steps  Of  The  System  Design  Process. 


KEY  ISSUES 

At  each  phase  of  the  program,  user  con¬ 
currence  is  required  and  is  vital  to  mis¬ 
sion  success.  The  system  requirements 
review,  conducted  during  the  conceptual 
phase,  is  one  of  the  most  important  mile¬ 
stones  in  the  design  review  process. 

The  life  cycle  cost  or  design  to  cost 
critical  requirement  for  cost  data  is  at 
the  preliminary  design  review:  if  several 
design  options  are  presented,  cost  consid¬ 
erations  play  a  vital  discrimination  role. 

Configuration  audits  are  vital  to  ensure 
a  firm  baseline,  properly  documented.  A 
physical  and  functional  audit  for  hardware, 
software  and  man-machine  interfaces  to 
verify  "A"  specification  compliance  is 
mandatory, 

RECOMMENDATION 

The  design  review  process  should  stress 
the  role  of  systems  engineering  and  user 
concurrence  at  all  steps  of  the  system 
design  process. 
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DESIGNING  FOR  MISSION  ASSURANCE 

MISSION  SUCCESS 
THROUGH  THE 

EARLY  DETECTION  OF  POTENTIAL  FAILURES 
VIA  THE  "PRINCIPAL  ENGINEER"  CONCEPT 

BY 

CHARLES  BIERMAN 
FIVE-YEAR  RECORD 

This  success  record  is  I'n  a  large  measure 
attributed  to  the  "principal  engineer" 
concept  and  formal  review  process.  The 
"principal  engineers"  are  top  system 
designers  and  analysts.  The  purpose  of 
the  review  process  is  to  provide  maximum 
assurance  that  the  launch  vehicle/space¬ 
craft/  software/GSE  at  critical  schedule 
points  are  free  of  problems,  mistakes, 
or  potential  failures.  The  principal 
engineers  are  the  key  "investigators"  who 
report  their  findings  to  meetings  of  joint 
government-contractor  management. 


FIVE-YEAR  RECORD 

39  SUCCESSFUL  SPACE  MISSIONS 


17 

17 

5 

ATLAS/CENTAUR 

ATLAS 

TITAN/CBNTAUH 

2  PIONEER  VENUS 

6  GPS 

2  VIKING 

3  HEAO 

1  P78-1 

2  VOYAGER 

3  FLTSATCOM 

1  TIROS 

1  HEUOS 

3  COMRTAR 

1 SEASAT 

6  INTELSAT 

1  NOAA-A 

7  MISC. 

1  UNSUCCESSFUL  ATLAS/CENTAUR  (INTELSAT)  IN  1977 


PRIMARY  CAUSES 
MARGINAL  DESIGNS 

Can  be  rectified  by  better  design  reviews 
and  adequate  qualification  level  testing. 

"INNOCENT"  CHANGES 

The  full  impact  of  the  change  on  the  system 
or  other  systems  is  not  fully  evaluated  or 
tested. 

INADEQUATE  TESTING 

Didn't  test  something  you  could  have 
tested.  The  space  system  must  work  per¬ 
fectly  the  first  time.  Thorough  check¬ 
out  of  hardware  and  software  is  mandatory. 


Testing  must  be  thorough  enough  to  detect 
marginal  designs  during  qualification  and 
system  performance  prior  to  launch. 

INADEQUATE  REVIEW  OF  DATA 

The  data  was  trying  to  tell  you  something. 
Too  often  flight  failures  cause  a  re-review 
of  test  data  that  had  a  message  which  was 
ignored  or  overlooked. 

POOR  QUALITY  AND  WORKMANSHIP 

The  only  failure  in  the  past  five  years 
falls  in  this  category.  Constant  effort 
is  required  to  maintain  quality  standards. 

PRIMARY  CAUSES  OF  A  MISSION  FAILURE  IN  ANY  PROGRAM 

PREMISE-  NO  ONE  WANTS  TO  MAKE  AN  ERROR.  HOWEVER,  FAILURES  CAN 
OCCUR  BECAUSE  OF  UNDETECTED  PROBLEMS  OR  MISTAKES  MADE  IN  THE 
FOLLOWING  AREAS: 

•  MARGINAL  DESIGNS  INADEQUATELY  PROOFED  AND  QUAUFIED 

•  "INNOCENT"  DESIGN/SOPTWARE  CHANGE? 

•  INADEQUATE  READINESS  TESTING 

•  INADEQUATE  REVIEW  OP  TEST  DATA 

•  POOR  QUALITY  AND  WORKMANSHIP 

FINDING  MISTAKES 

A  positive  attitude  by  both  government  and 
contractor  management  is  of  vital  impor¬ 
tance  in  the  problem  resolution  cycle. 

Good  managers  should  encourage  the  engineers 
in  discovering  any  mistakes  and  help  them 
gather  resources  necessary  to  correct 
the  problems. 

Engineers,  assemblers,  technicians,  etc., 
are  all  human  and  can  make  mistakes.  We 
must  promote  the  attitude  in  all  partici¬ 
pants  to  be  constantly  alert  to  weaknesses 
and  potential  failures.  All  people  involved 
should  be  encouranged  to  come  forth  with 
their  worries,  problems,  or  goofs  without 
fear  of  reprisal,  etc. 

FINDING  MISTAKES.  PROBLEMS.  OR  POTENTIAL  FAILURES 

•  POSITIVE  APPROACH  -  MANAGEMENT  AND  GOVERNMENT 

A  HUMANS  DO  MAKE  MISTAKES 

A  ENCOURAGE  PEOPLE  TO  RECOGNIZE  AND  ADMIT  THOR  OWN  MISTAKES 
IN  A  TIMELY  FASHION 

A  COMMEND  THOSE  WHO  DISCOVER  MISTAKES  OR  POTENTIAL  FAILURES 
A  NO  PENAL'nES  OR  REPRISAIB  FOR  THOSE  WHO  MAKE  A  MISTAKE 
A  WHOLEHEARTED  MANAGEMENT  SUPPORT  IN  CORRECTING  MISTAKES 

•  REPRISAL  APPROACH  IS  NEGATIVE 

A  MISTAKES  CAN  GO  UNREPORTED 
A  MISTAKES  MAY  BE  COVERED  UP 
A  MISTAKES  MAY  NOT  BE  DI800VERED  IN  TIMC 


¥ 


i 


I 
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PRINCIPAL  ENGINEER  CONCEPT 

Outstanding  designers/analysts  are  selected 
to  fill  the  roles  of  principal  engineers. 
They  must  be  of  a  caliber  whose  judgment 
and  recommendations  you  respect,  whom  you 
are  proud  to  have  on  your  team,  and  who 
act  as  an  extension  of  yourself. 


•  QUAUFICATIONS 

A  TOP  DESIGNERS/ANALYSTS 

A  HIGH  IN  rNTTlATIVE,  INTEGRITY.  AND  TEAM  SPIRIT 
A  RESPECTED  BY  PEERS.  MANAGEMENT,  AND  GOVERNMENT 

•  RESFONSIBIUTIES 

A  ALERTNESS  TO  SYSTEM/COMPONENT/SOFTWARE  WEAKNESSES 

A  SUPPORT  PROBLEM  INVESTIGATION  DURING  MANUFACTURE, 
TEST,  AND  LAUNCH 

A  PROVIDE  RECOMMENDATIONS  FOR  PROBLEM  RESOLUTION 
A  SUPPORT  FORMAL/INFORMAL  REVIEW  PROCESSES 


•  MANAGEMENT'S  NURTURING  OF  THE  PRlNaPAL  ENGINEER  CONCEPT  BY  LISTENING 
TO  HIS  RECOMMENDATIONS  AND  JUDGMENTS  PRIOR  TO  MAKING  A  CRITICAL 
DECISION  IS  THE  KEYSTONE  OF  A  SUCCESSFUL  PROGRAM. 

•  MANAGEMENT'S  REACTION  TO  TIMELY  REPORTED  ERRORS  SHOULD  BE 
CHARACTERIZED  BY  GRATITUDE  AND  ASSISTANCE  IN  RESOLUTION,  NOT  OF 
REPRISAL  AND  FINGER  POINTINC. 


FORMAL  REVIEW  PROCESS 


AAAAAAA  A  AA 

•  FOR  •CM 


LAUNCH  TECHNICAL 
READINESS  SUPPORT 
REVIEWS  DURING 

COUNTDOWN 


OO-AHEAO  LAUNCH 

•  PRESENTLY  DEFINED  IN  MIL<ETD-1621A 


DESIGN  PRODUCT  SYS.  OUAL. 

REVIEWS  REVIEWS  REV.  OlSCREP. 

REVIEW 


DESIGN  REVIEWS.  SYSTEMS  REVIEWS.  QUALITY 


DISCREPANCY  REVIEWS 


These  milestone  formal  reviews  are  con¬ 
ducted  as  joint  government-contractor 
meetings.  SPDRs,  PDRs,  and  CDRs  are 
defined  review  milestones  from  MIL-STD- 
1521 A. 


The  objective  of  Systems  Reviews  is  to 
ensure  than  the  launch  vehicle  is  con¬ 
figured  to  satisfy  mission  requirements. 
All  mission  analysis,  subsystem  hardware 
design,  and  software  design  are  assessed 
for  compatibility  with  each  other  and  the 
current  mission  definitions. 


The  objective  of  the  Quality  Discrepancy 
Reviews  is  to  assure  the  integrity  of 
flight  hardware  by  a  review  of  quality 
discrepancy  dispositions. 


•  THE  PRlNaPAL  ENGINEeR.  BY  MANAGEMENT'S  ACTION,  KNOWS  HE  IS  BEING  HELD 
RESPONSIBLE  FOR  HIS  HARDWARE.  KIS  OWN  PRIDE  WILL  ASSURE  HE  IS  NOT 
EMBARRASSED  BY  ANY  POOR  DESIGN,  HARDWARE  FAILURE,  OR  A  SCHEDULE  DEIAY. 

•  MANAGEMENT  MUST  BACK  UP  THE  PRlNaPAL  EHGINECR'S  ROLE  IN  RESOLVING 
POTENTIAL  FAILURES  AND  CORRECTING  MISTAKES  (REGARDLESS  OF  WHO  MADE  THEM). 


REVIEW  CYCLE 

A  planned  and  funded  program  of  reviews 
involves  the  government  and  the  principal 
engineers  at  each  major  step  of  space 
system  development  from  concept  through 
launch. 

The  following  photos  show  the  review 
process  in  action. 

These  planned  reviews  are  supplemented  by  FACTORY  PRODUCT  REVIEWS 
problem  solving  reviw  boards  which  meet  Following  the  release  of  drawings,  the 

as  required.  The  principal  engineer  is  procurement/production/testing  cycle 

usually  the  key  participant  in  these  begins.  This  is  where  weaknesses  in  the 

reviews.  design  and  manufacturing  errors  surface. 


LAUNCH  SUPPORT  TEAM 


So  we  get  the  principal  engineers  directly 
involved  in  the  manufacturing/testing 
cycle.  Their  formal  approval  at  key 
assembly  points  must  be  obtained  before 
work  may  continue.  The  final  review  is 
the  completed  launch  vehicle. 

Review  sequence: 


1.  Quality  Control  Inspection 

2.  Principal  Engineer  Review 
3..  Government  (DCAS)  Review 


LAUNCH  READINESS  REVIEW  AT  THE  LAUNCH  SITE 


SITE  REVIEW  "TEAM"  CONSISTS  OF; 

1.  Government  (plua  consultants) 

2.  Principal  Engineers 

3.  Launch  Site  Engineers 

4.  Associate  Contractors 
REVIEW  ITEMS: 

1 .  Walkdown 

2.  Data 

3.  Procedure  History  Data 

4.  Site  Configuration  Control  Paper 

5.  Quality  Assurance  Records 


REVIEW  RESULTS: 

The  results  of  this  review  are  presented  to  Government,  Convalr, 
and  associate  contractor  management. 


AGAIN  THE  '*TEAM"  IS  COMPOSED  OF: 

1.  Government  (plus  consultants) 

2.  Principal  Engineers 

3.  Launch  Site  Engineers 

4.  Associate  Contractors 

After  years  of  work,  success  can  be  determined  by  the  few 
hours  of  countdown  and  booster  flight.  If  all  responsible  parties, 
including  the  principal  engineers,  have  done  their  job  well,  success 
is  greatly  enhanced. 

In  this  photo,  the  team  is  mcmitoring  the  countdown  event.  They 
are  ready  to  help  resolve  problems  that  arise  during  the  countdown. 

T-5  MINUTES  AND  COUNTING 
ALL  SYSTEMS  "GO" 


LAUNCH  SUPPDKJ  TCAM  T -5  MiN  AND  COUWTING 


CONCLUSIONS  AND  RECOMMENDATIONS 

Use  of  the  approaches  presented  has  been 
a  key  element  in  the  successful  record 
achieved  on  space  programs  contracted  by 
Convair. 

These  and  similar  approaches  are  used  in 
other  programs. 

For  space  applications,  the  principal 
engineer  concept  can  be  a  powerful  tool  for 
increasing  launch  success  probability. 


•  THE  PRINCIPAL  ENGINEER  CONCEPT  AND  THE  REVIEW  PROCESSES  SHOWN  BELOW  ARE 
NOT  NOW  IN  MIUrrD-)»JA  fUSAP). 


•  THEIR  INCLUSION  WOUU)  BENEFIT  THE  GOVERNMENT; 

A  IMPROVED  VTSIBIUTY  AS  TO  MISSION  READINESS 
*  REAL  TIME  INVOLVEMENT  IN  ACTION  ITEM  DECISIONS 
A  MUCH  IHCRBASED  PROBABIUTY  OF  MISSION  SUCCESS 


•  IMPlfMENT  THE  PRINCIPAL  ENGINEER  CONCEPT  AND  P*  OB  IN  THE  MILITARY 
STANDARD  AS  NORMAL  PROCESSES  FOR  SPACE  PROGRAM 

A  FACTORY  PRODUCT  REVIEWS 
A  SYSTEMS  REVIEWS 
A  QUALITY  OCSCREPANCIES  REVIEWS 
A  UUNCH  STTE  PRODUCT  ft  READINESS  REVIEWS 

•  RECOGNIZE  THE  FUNDING  REQUIREMENTS  TO  ACCOMPLISH  THESE  REVIEWS. 


AGENDA 


DESIGNING  FOR  THE  SHUTTLE  ERA. 


INTRODUCTION  AND  OVERVIEW  BY  THE  SUB-GROUP 


oesiai  WBICSHCP  -  -DESIGldltG  FOR  SHUniE  ERA" 


CHAIRMEN 

J.B.  Sterett 

Marshall  Space  Flight  Center 
NASA 


DESIGNING  FOR  THE  SHUTTLE  ERA  MISSION  ASSURANCE 


-  INCLUDES  CONSIDERATIONS  OF  PAYLOAD  WEIGHT, 
COST,  AND  SCHEDULE  AS  WELL  AS  FUNCTIONAL 


tNTACDUCTIOi 

0  WORKSHOP  APPROACH 
0  KEY  ISSUES/OISCUSSICN  TOPICS 
O  WORKSHOP  "WOUS  OPERANDI" 

DESIGN  REQUIREMENTS 

0  HOW  OEflHED/COr.'TROLLED 
0  HOW  GOVERNMENT  EXPECTS  REQUIREMENTS  UTILIZED 
O  COICERNS 


COITRACTOR  VIEWS  RELATIVE  TO  DISCUSSICN  TOPICS 
0  ORBITER 
0  lUS  BOOSTER 
0  SPACE  TELESCOPE 
O  CARGO  INTEGRATION 


DISCUS'SICN 

O'  REDEFINE  ISSUES 
o  SUMMARIZE  POSITIONS 
0  DISCUSS  DIFFERENCES 
0  $UKy.ai2E  RECOMMENDATIONS 


Q.  STERETT,  HASA/msfc 


L.  WILLIAMS,  NASA/.1SC 
e.  WAGNER.  AEROSPACE 


H.  EMICH,  ROCKWELL 
J.  ECKLE,  BOEING 
M.  MERCER,  LOCKHEED 
P.  CHRISTENSEN  ,  MARTIN 

J.  STEREH,  NASA/MSFC 


PERFORMANCE 


SPLINTER  KEETINGS 

0  ACTIVATED  IF  MORE  TIME  NEEDED  TO  WORK  RECOWENDATIOIS. 


SHUTTLE  ERA  "UNIQUE"  DESIGN  CONSIDERATIONS 


WORKSHOP  MODUS  OPERANDI 


-  SHUTTLE  ELEMENTS  DES TONED  AND  TESTED 
CONCURRENTLY  WITH  DESIGN  AND  TESTING  OF 
VARIOUS  PAYLOADS 

-  PAYLOADS  REQUIRED  TO  DESIGN  FOR  ORBITER  LANDING 
LOAD  CONDITIONS 

-  SHUTTLE  PROVIDES  SERVICING,  REPAIR,  AND  RECAPTURE 
OF  PAYLOADS  IN  EARTH  ORBIT 


0  QUESTIONS  TO  SPEAKERS  PERMITTED  FOR  UNDERSTANDING 
OR  CLARIFICATIONS 

0  SPEAKERS  WILL  PARTICIPATE  AS  RESPONDERS  TO 
QUESTIONS/COMMENTS 

0  DISCUSSION  PERIOD  QUESTIONS/COMMENTS  FROM  WORKSHOP 
AUDIENCE  IN  WRITING 

0  NOT  A  SHUTTLE/ORBITER  PROGRAM  REVIEW 


"DESIGNING  FOR  THE  SHUTTLE  ERA" 


J.  B.  STERETT 


THEME:  ASSURING  MISSION  SUCCESS  IN  DESIGNING  FOR  TRANSPORT  ON  THE  SHUTTLE. 

KEY  DISCUSSION  TOPICS 

1.  STRUCTURAL  DESIGN  REQUIREMENTS 

LOADS  ARE  CHANGING  -  TIMELINESS  OF  INFORMATION  -  HCW  MUCH  CONSERVATISM  IS  BEING  DESIGNED  IN?  -  IS  IT 
REALISTIC?  -  IS  IT  IMPACTING  MISSION  ASSURANCE? 

2.  PREDICTED  ENVIRONMENTS  -  (ACOUSTICS/VIBRATION/THERMAL.  ETC.) 

CHANGING  REQUIREMENTS  -  TIMELINESS  OF  INFORMATION  -  DATA  FLOW  I^ND  INTERCHANGE  -  HOW  CONSERVATIVE  ARE 
THE  REQUIREMENTS?  -  HOW  MUCH  CONSERVATISM  IS  BEING  DESIGNED  INTO  THE  HARDWARE?  MISSION  ASSURANCE 
IMPACTS? 

3.  MEASURED  ENVIRfflMENTS 

WHEN  AND  TO  WHAT  EXTENT  WILL  THE  SHUTTLE  FLIGHT  ENVIROIHENT  RESPONSE  BE  MEASURED?  WHAT  IS  THE  PLAN  TO 
DISSEMINATE  THE  RESULTS  TO  THE  PAYLOAD  DESIGNERS?  HOW  Wia  THE  DATA  BE  USED  BY  THE  PAYLOAD  DESIGNERS? 
AFFECTS  ON  MISSION  ASSURANCE? 

4.  DIFFERENCES  IN  AF  AND  NASA  SPECIFICATIONS,  DESIGN  CONSTRAINTS  AND  DESIGN  PHILOSOPHY 

IS  THIS  GENERATING  PROBLEMS  AND  WHAT  IS  THE  IMPACT  ON  MISSION  SUCCESS?  ARE  WE  ACHIEVING  MISSION 
ASSURANCE  WITH  THE  SPECIFICATIONS  THAT  WE  ARE  USING  TODAY?  RECOIMENDATIONS  FOR  CHANGES? 

5.  SAFETY  REQUIREMENTS 

WHAT  IS  THE  IMPACT  ON  DESIGN?  HCW  WILL  INTERAGENCY  APPROVAL  AND  CERTIFICATION  BE  OBTAINED? 

6.  MULTIPLE  PAYLOAD  CARGO 

INTERACTION  OF  PAYLOADS  AND  IMPT,*  ON  MISSION  ASSURANCE. 

7.  REUSEABILITY  REQUIREMENTS 

IMPACTS  ON  DESIGN  FOR  MULTIPLE  MISSIONS  -  ARE  REQUIREMENTS  WELL  DEFINED  (MIL-STD-1 540?)? 

IMPACTS  ON  MISSION  ASSURANCE? 


DESIGNING  FOR  THE  SHUTTLE  ERA 
PAYLOAD  REOUIREilENTS  DEFINITION  AND  IMPLEMENTATION 


L.  G.  WILLIA.MS  NASA  -  JOHNSON  SPACE  CENTER 


APRIL  28.  1980 


DESIGNING  FOR  THE  SHUTTLE  ERA  GENERAL 
REQUIREMENTS  DEFINITION 

This  chart  summarized  the  Shuttle  Programs 
approach  to  the  definition  of  payloads 
accommodations  and  design  requirements. 

It  is  important  to  note  that  ICD  2-19001 
defines  all  of  the  payload  interfaces. 

Most  users  require  only  a  fraction  of  these 
accommodations  and  the  standard  allocation 
of  these  accommodations  is  defined  in 
Volume  XIV.  ICD  2-19001  is  used  as  the 
basis  for  payload  specific  ICD's.  Each 
paragraph  (or  accommodation)  is  disposi- 
tioned  as  applicable,  not  applicable, 
exception  or  unique  application.  The 
intent  is  to  always  sign  up  the  user  to  a 
set  of  interfaces  equal  to  or  less  than 
the  Siiuttle  accommodations.  Thus  avoiding 
unique  modifications  to  the  reusable 
Shuttle  vehicle.  The  ICD  2-19001  is  a 
contract  requirement  on  the  payload 
integration  contract  only.  This  is  a 
unique  approach  as  opposed  to  the  expend¬ 
able  booster  ICD's  where  a  booster  could 
be  easily  modified  to  the  specific  pay- 
load  requirements. 


DESIGNim  FOR  THE  SHUHLE  ERA 
GENERAL  REOUIRENENTS  OEFINITIOH 

0  SPACE  SHUTTU  SYSTEH  PAYIOAO  ACCOWODATIONS  -  OOCUItKT  JSC  07700.  VOLIK  XIV 
PBIWIllfS;  CAPABILITIES  Of  SPACE  SHUTTLE  SYSTEH  TO  ACCOWODATE  PAYLOADS  GENERAL 
DESCRIPTION  Of  SPACE  TRANSPORTATION  SYSTEH  AND  PLOH  DESIGN  CRITERIA 
FOR  DEDICATED  USERS  AND  HIXED  USERS 

CONTROL:  HAINTAIttD  AND  CONTROLLED  BY  SPACE  SHUTTLE  PROGRAH  OFFICE 

0  SHUTTLE  ORBITER/CARGO  STANDARD  INTERFACES  -  ICD  2-1900L 

PROVIDES:  DEFINITION  AND  CONTROL  Of  SERVICES  PROVIDED  BY  THE  SHUTTU  SYSTEH 
AT  THE  USER  INTERFACE. 

DESIGN  INTERFAUS  FOR: 

0  NECHANICAL 
0  STRUCTURAL 
0  THERHAL 
0  POHER 
0  AVIONICS 
0  SOFTWARE 

DEFINES  INDUCED  ENVIRONHENTAL  INTERFAUS 
DEFINES  GENERAL  INTERFAU  CONSTRAINTS  AND  LIHITATIONS 
CONTROL:  HAINTAINED  AND  CONTROlUD  BY  SPAU  SHUTTU  PROGRAH  OFflU 


DESIGNING  FOR  THE  SHUTTLE  ERA  GENERAL 
REQUIREMENTS  DEFINITION 

NHB  1700.7,  Safety  Policy  and  Requirements 
for  Payloads  using  the  STS  represents  a 
set  of  new  requirements  for  payloads  as 
opposed  to  those  for  expendable  boosters. 
They  are  necessitated  by  the  presence  of 
the  flight  crew  and  their  exposure  to 
hazards  for  the  attached  phases  of  the 
mission.  The  responsibility  for  payload 
safety  rest  with  the  user  and  the  NASA 
safety  panel  reviews  the  user  provides 
analysis  to  assure  understands  the 
requirements.  It  does  not  perform  detailed 
review  and  approval  of  the  design  and 
verification  process  but  maintains  an 
advisory  and  overview  of  the  payload 
safety  process. 

JSC  14046,  Payload  Interface  Verification 
Requirements  is  a  new  document  defining 
the  payload  and  STS  interface  verification 
requirements.  It  provides  an  overview  of 
the  STS  interface  verification  process  in 
preparation  for  integrated  STS/Payload  o- 
perations,  defines  payload  interface  veri¬ 
fication  requirements  and  defines  integrat¬ 
ed  STS/Payload  test  objectives.  Again  the 
responsibility  is  placed  on  the  user  to  de¬ 
fine  and  execute  his  verification  program. 


DESIGNING  FOR  THE  SHUTUE  ERA 
GENERAL  REDUIREHENTS  DEFINITION 

0  SAFETY  POLICY  A«D  REDUIREHENTS  FOR  PAYLOADS  USING  THE  STS  -  DOCUHENT  HHB  1700.7 

PRQVIDFS:  POLICY  AND  HINIWH  SAFETY  REOUIRE’IENTS  APPLICABLE  TO  ALL  STS 

PAYLOADS,  AIRBORNE  SUPPORT  EOUlPftNT  AND  GROUND  SUPPORT  EQUlPftNT 

tONTROt ;  .-WINTAINED  AND  CONTROLLED  BY  NASA  HEADOU.ARTERS 

JSC  SAFETY  PANEL  ACTS  ON  WAIVERS  BUT  DOES  NOT  MODIFY  REOUlREltNTS 


0  PAYLOAD  INTERFACE  VERIFICATION  REfiUIREMENTS  -  DOCUMENT  .ISC  140N6 

PROVIOFS:  BASIC  REOUIREMENtS  TO  BE  UTILIZED  BY  THE  STS  AND  THE  STS  USER 
IN  THE  VERIFICATION  OF  PAYLOAD  AND  PAYLOAD  GROUND  AND  AIRBORNE 
SUPPORT  EQUIPMENT  INTERFACES  WITH  THE  STS. 

CONTROL:  MAINTAINED  BY  JSC  SHUTTLE  PAYLOAD  INTEGRATION  DEVELOPftNT 

OFFICE  AND  KSC  CARGO  PROJECTS  OFFICE. 

controlled  by  SHUTTLE  PROGRAM  OFFICE  AS  PART  OF  JSC  07700.  VOLUME  XIV 


DESIGNING  FOR  THE  SHUTTLE  ERA  CONCERNS 

The  NASA  recognizes  that  designing  for 
the  Shuttle  imposes  new  requirements  and 
risk  on  the  payloads.  Hopefully  this  is 
offset  by  the  additional  capabilities  and 
economy  of  using  the  system.  The  con¬ 
current  development  of  the  STS  and  payloads 
creates  a  risk  in  designing  to  presently 
defined  environments.  My  personal  assess¬ 
ment  is  that  conservation  on  the  users 


part  at  this  point  In  time  Is  desirable. 

For  Instance  a  contamination  cover  may  be 
cost  effective  and  provide  mission  assur¬ 
ance  as  compared  to  expensive  analysis 
and  low  level  of  confidence  at  this  point 
In  time. 

The  length  of  time  of  attached  operations 
for  Shuttle  payloads  far  exceeds  that 
previously  experienced  on  expendable  launch 
vehicles.  This  Imposes  a  new  set  of  design 
requirements  on  the  payload  as  well  as 
providing  additional  capability  for  mission 
assurance  checkout  In  orbit  prior  to  com¬ 
mitting  to  freefllght  operations.  We 
believe  this  Is  an  area  the  user  must 
recognize  early  In  his  development  process. 
Incorporate  Into  his  design  'requirements 
and  utilize  to  his  advantage. 

The  average  user  has  not  fully  recognized 
the  difference  In  the  Shuttle  program 
and  expendable  launch  vehicles.  Many 
are  reluctant  to  accept  standardization 
and  design  to  the  mixed  user  criteria. 

They  fall  to  recognize  that  It  Is  Intended 
to  maximize  mission  assurance  as  well  as 
minimize  costs.  The  best  way  to  use  a 
Shuttle  Is  exactly  as  It  was  used  success¬ 
fully  on  the  previous  mission. 


DESIGNING  FOR  THE  SHUTTLE  ERA 
CONCERNS 

0  CONCURRENT  OEVELOPliEHT  OF  PAYLOADS  AND  SPACE  TRANSPORTATION  SYSTEN 

-  CREATES  RISK  IN  DESIGNING  TO  PRESENTLY  DEFINED  INDUCED  ENVIRONMENTS 
LOADS 

ACOUSTICS 

THERMAL 

CONTAMINATION 

-  REQUIRES  USE  OF  CONSERVATIVE  DESIGN  PRACTICES 

0  SHUTTLE  DESIGN  AND  OPERATIONS  HILL  NECESSITATE  MORE  ON-ORBIT  TIME 
PRIOR  TO  DEPLOYMENT 

-  RESULTS  IN  additional  CONSTRAINTS  WITH  RESPECT  TO  THERMAL. 

POWER  AND  COIITAMINAIION 

0  SOME  PAYLOADS  DEMAND  PREFERENTIAL  TREATMENT  WITH  RESPECT  TO  LOCATION. 
PROVISION  OF  NON-STANDARD  SERVICES  AND  FIRST  DEPLOYMENT 

-  REQUIRES  EXTENSIVE  NEGOTIATION 

-  CONSTRAINING  TO  MANIFESTING  OF  CARGOES  BY  MISSION 

-  RESULTS  IN  SIGNIFICANT  CUMULATIVE  PRODUCTIVE  TIME  LOSS  FOR  ALL  INVOLVED 

-  SPECIAL  INTERFACES.  DEVIATING  FROM  THE  STANDARD.  INCREASE  PROBABILITY 

OF  ANWIALY  OCCURRENCE  THEREBY  POTENTIALLY  AFFECTING  MISSION  SUCCESS 


STS/PAYLOAD  REQUIREMENTS  DEFINITION 

documentation 

This  chart  Illustrates  the  relationship  of 
the  various  requirements  documentation  In 
the  STS  payload  Integration  process.  We 
have  previously  discussed  ICD  2-19001  and 
the  payload  specific  ICD's  of  particular 
note  Is  the  Payload  Integration  Plan  (PIP) 
which  Is  used  by  JSC  as  a  program  level 
Integration  "statement  of  work",  top 


level  Interface  definition  and  Integration 
activities  schedule  agreement.  Major 
constraints  on  mission  design  and  operation 
are  defined,  the  launch  sight  operations 
are  summarized,  and  all  non-standard 
(optional)  services  are  agreed  In  this 
document.  It  constrains  the  payload 
specific  ICD  development  and  PIP  annexes 
which  are  user  supplied  data  for  mission 
Implementation. _ 

STS/PAYLOAD  REQUIREMENTS  DEFINITION  DOCUMENTATION 


SHUTTLE  payload  INTEGRATION  PROCESS 

This  chart  Illustrates  the  time  frame  for 
the  overall  payload  Integration  process. 
Note  that  the  requirements  development 
Is  driven  by  the  payload  schedules.  They 
should  however  be  well  defined  by  2  years 
prior  to  flight  In  order  to  support  an 
orderly  integration  process. 


SHUTTLE  PAYLOAD  INTEGRATION  PROCESS 
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Designing  for  the  Shuttle  Era- 
Structural  Design  for  DDD  Payloads 


Methods  of  Structural  Verification 


THE  AEROSPACE  CORPORATION 
R.  G.  WAGNER,  PROJECT  ENGINEER 


Overview 

•  BASIC  STRUCTURAL  DESIGN  REQUIREMENTS 

•  METHODS  Of  STRUCTURAL  VERIFICATION 

•  STRUCTURAL  DESIGNTTEST  CRITERIA  OPTIONS 

•  FRACTURE  CONTRa 

•  SAFETY  REVIEW  PROCESS  AND  CERTIFICATION 

•  SHUHLE  STRUCTURAL  LOADS  ENVIRONMENT 

•  LOW  FREQUENCY  REGIME  HRANSIENT  RESPONSE) 

•  HIGH  FREQUENCY  REGIME  (ACOUSTICS) 

•  ENVIRONMENT  VERIFICATION  PROGRAM 

•  GROUND  TESTS 
•FLIGHT  TEST  DATA 

•  CONCLUDING  REMARKS 


Basic  Structural  Design  Requrements 


•  NASA 

•  SHUTTLE  CREW  AND  OR8ITER  SAFETY 

.  SUSTAIN  ALL  CRITICAL  LOADS  WITH  A  FACTOR  OF  SAFETY  >  1.4 
.  LIFE  VERIFICATION  AND  FRACTURE  CONTROL 


•  AFSD 

•  MISSION  SUCCESS 

.  SUSTAIN  ALL  CRITICAL  LOADS  WITH  A  FACTOR  OF  SAFETY  ?  I.2S 
.  COMMANDER'S  POLICY  REQUIREMENTS 

.  INDEPENDENT  LOADS  ANALYSES 

.  TEST  VERIFIED  STRUCTURAL  DYNAMIC  MODELS 

.  FRACTURE  MECHANICS  ANALYSES  OF  PRESSURIZED  STRUCTURES 

•  SHUTTU  CREW  AND  ORBITER  SAFETY 


•  OBJECTIVE;  DEMONSTRATE  COMPLIANCE  WITH  BASIC  REQUIREMENTS 

•  PRIMARY  STRUCTURE 

.  STRENGTH  -  STRUCTURAL  DESIGN/TEST  CRITERIA  OPTIONS 

.  DEFORMATION  -  ANALYSIS,  FOR  ClEARANCE  AND  TRUNNION 
PULL-OUT 

.  INTERFACE  LOAD  COMPATIBILITY  -  ANALYSIS,  FOR  COMPATIBILITY 
WITH  ORBITER  CAPABILITY 

.  STRUCTURAL  DYNAMIC  CHARACTERISTICS  -  TEST  VERIFICATION 

.  LIFE  VERIFICATION  AND  FRACTURE  CONTROL  -  INSPECTION  AND 
ANALYSIS  llracture  control  plan  currently  under  review! 

•  SECONDARY  STRUCTURE 

.  COMPONENT  VIBRATION  QUALIFICATION  TESTS 
.  SYSTEM  LEVEL  ACOUSTIC  TESTS 


SIniclinl  DeUpi/TMl  Crilerii  OpUm  far  DOD  Piyloadi  on  Shuttle 


•  OPTIONS 


OPTION 

TEST 

LEVEL* 

FACTORS  Of  SAKTY 
YIELD  ULIIMAIE 

TEST  SUCCESS 
CRITERIA 

TYPICAL 

APPIICAIION 

1.  TEST  TO  ULIIMAIE 
(Basallna) 

1.4 

I.O 

1.4 

•  NO  FAILURES  11.4) 

•  NO  DETRIMENTAL 

DEFORMATION  (1.0) 

llEEi 

2.  REDUCED  LEVEL  TEST  ON 
FLIGHT  ARTICLE 

1.2S 

l.» 

1,4** 

•  NO  DETRIMNIM 
DEFORMATION 

SMALL  FLEET 

3.  PROOF  TEST  EACH  FLIGHT 
ARTICLE 

1.1 

I.l 

1.4 

•  NO  DETRIMENTAL 
DEFORMATION 

KW 

4.  NO  SYSTEM  STATIC 
QUMIfiCAIION  TEST 

NA 

1.6 

2.25 

NA 

ONE-OF-A-KIND. 
MODS  10  EXISTING 
SIC 

mulUptytng  limit  lead 

Note  the  eddlUonel  analysis  raquiramant  balow 
ANALYSIS  REQUIREMENTS 

•  POSITIVE  MARGINS 

•  FOR  OPTION  2.  MINIMUM  MARGINS  FOR  STABILITY  CRITICAL  STRUCTURE 

•  APPLICABILITY:  SAFETY -CRITICAL  STRUCTURE  OM-Y.  WITH  REQUIREMENTS  REDUCED  TO  MISSION 

SUCCESS  LEVEL  FOR  NON  SAFETY -CRITICAL  STRUCTURE 

•  STATUS 

•  COMPATIBLE  WITH  CURRENT  SPACECRWT  APPROACHES  AND  PAST  AFSD  PRACTICES 

•  NASA/ JSC  POSITION 

•  CONCURRENCE  WITH  ALL  OPTIONS 

•  DEVIATIONfCONCURRENCE  REQUIRED  IN  CLASSIFYING  STRUCTURE  AS  "NON-SAFETY -CRITICAL" 


SUSTAIN  ALL  CRITICAL  LOADS  WITH  A  FACTOR  OF  SAFETY  f  1.4 
LIFE  VERIFICATION  AND  FRACTURE  CONTROL 


Fracture  Control  Approach  (under  review) 


Shuttle  Stnictinl  Loads  Emdronmiiit  (Confd) 


•  HIGH  FREQUENCY  REGIME  LMDS-  ENGINE  ACOUSTIC  NOISE 

•  NASA  ICD  2  \Wt  SPEC  LEVEL  BASED  ON  TITAN  SPECS 

•  CARGO  BAY  ACOUSTIC  PREDICTION  MADE  ON  BASIS  OF  LIMITED  GROUND  TESTS 

•  A4%  MODEL  DATA  USED  FOR  BASIC  ENVIRONMENT  DEFINITION 

•  FULL  SCALE  NOISE  REDUCTION  TESTS  ON  ORBITER 

•  QUARTER-SCALE  MODEL  TESTS 

•  MODEL-TO-FUIL  SCALE  COMPARISON  DATA 


•  PAYLMD  PRESENCE  AFFECTS  CARGO  BAY  LEVELS 

•  CONTROVERSY  OVER  LuW  FREQUENCY  LEVELS  WHOSE  RESOUTION  REQUIRES 

FLIGHT  DATA 


•  f7«e«dwr«  Control 
o  fdkrlcocloo  Control 


9r«rlao  KooulfMont* 
o  !loc«riAlt  iml  ?roe««« 
SoocifUoe  tool 
•  Itrroo  Motsitudo 
0  now  Slio  thjoo  4M 

OtlOOCdClOO 
Froccort  Cjocrol 


SiMy  Ravlaw  Procats  and  Caitificatkin 


•  DOO  STS  SAFETY  REVIEW  TEAM  (SRT)  CONDUCTS  FOUR  DETAILED  TECHNICAL  REVIEWS 

•  FUNCTION  IS  TO  ENSURE  THAT  DOO  STS  SAFETY  REQUIREMENTS  ARE  INCORPORATED 

INTO  PAYLMD  DESIGN 

•  TYPICAL  SRT  MEMBERSHIP:  YVAS  ACCIDENT  PREVENTION  MANAGER  OR  PAYLMD 

ENGINEERING  OFFICER.  AEROSPACE.  SAMTQ  NASA-KSC.  NASA-JSC/SPIOPO. 

PIC.  BOEING  (lUS  PAYLMDS  ONLYI.  SD/SE  lADVISOR) 

•  SRT  PRESENTS  RECOMMENDATIONS  ON  FINAL  SAFETY  CERTIFICATION  TO  DOO  STS 

SAFETY  CERTIFICATION  PANEL  (SCPI 

•  SCP  MEMBERSHIP:  SAMTO.  SDVSE.  NASA  THDQTRSI,  AEROSPACE 

•  SCP  MAKES  RECOMMENDATION  FOR  CERTIFICATION  TO  DOO  STS  SPO 

t  DOO  STS  SPD  AND  PAYLMD  5PD  SIGN  CERTIFICATE  OF  SAFETY  COMPLIANCE  (CSCI 

•  CSC  PRESENTED  TO  NASA  (SPIOPOl  IN  ACCORDANCE  WITH  THE  OOOVNASA  PAYLMD 

CERTIFICATION  AGREEMENT 


•  THREE  LOAD  CYCLES 

•  PRELIMINARY  -  SUPPORTS  SPACECRAn  PRELIMINARY  DESIGN  REVIEW  IPDRI 

•  FINAL  DESIGN  -  SUPPORTS  SPACECRAH  CRITICAL  DESIGN  REVIEW  (CDR) 

AND  NASA  CARGO  INTEGRATION  REVIEW  ICIRI 

•  VERIFICATION  -  FINAL  PRERIGHT  VERIFICATION  OF  STRUCTURAL  ADEQUACY 

•  UNCERTAINTY  FACTORS  APPLIED 

•  MATURITY  OF  BOOSTER  MODaS  AND  FORCING  FUNCTIONS 

•  MATURITY  OF  SPACECRAn  SYSTEM  MODaS 

•  CURRENT  VALUES  FOR  TYPICAL  SPACECRAn  PROGRAM 

PRaiMINARY  FINAL  DESIGN  VERIFICATION 

SPACECRAn  1.5  1.25  l.O 

tUS  1.5  11.51  (1.5) 

SHUTTLE  1.5  (1.5)  (1.5) 

•  INDEPENDENT  CONTRACTOR  EFFORT 

•  VERIFICATION  CYCa  LOADS  ANALYSIS 

•  CONCURRENCE  WITH  TEST  VERIFICATION  OF  DYNAMIC  MOOa 


Maximum  Predictad  SPL  with  A  Smal  (  60%  diameter)  Payload 


Shuttle  Structural  Loadi  Environment 


»  LOW  FREQUENCY  REGIME  LOADS  -  DERIVED  FROM  COUPLED  SHUnaVPAYLOAD  DYNAMIC 
AND  QUASISTATIC  ANALYSES  (load  cycits) 


•  SHUTTU  DYNAMIC  MOOaS  AND  FORCING  FUNCTIONS  CFP  FROM  NASA 

o  SPACECRAn  DYNAMIC  MDOaS  AND  LOAD  TRANSFORMATION  (MATRICES 
FROM  SPACECRAn  CONTRACTORS 

O  TRANSFER  STAGE  (a.Q..  lUS.  PAM-0)  DYNAMIC  MOOaS.  FORCING 
FUNCTIONS  Alto  LTM*s  FROM  CONTRACTOR 

•  ANALYSIS  METHOOOIXY  FOR  EACH  SPACECRAn  SYSTEM 
o  LIFTOFF  AND  LANDING  LOADS  BY  DYNAMIC  ANALYSIS 

o  QUASISTATIC  LOADS  ANALYSES  FOR  OTHER  EVENTS  Igust,  angina 
shuMawns.  stagings.  alc.L  INCLUDING  THERMAL  EFFECTS 
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Conckiing  Commants 


•  BASIC  APPROACH  OF  AN  INTFCRAUD  PROGRAM  OF  OCSICN.  ANALYSIS.  AND  TEST 

USING  THE  SAME  CONCEPTS  AS  EMPLOYED  ON  EXPENDABLE  BOOSTERS  *  LMD 
CYCLES.  UNCERTAINTY  FACTORS.  MODAL  SURVEYS.  STATIC  TESTS,  AND  ACOUSTIC 
TESTS-  IS  GENERALLY  APPLICABLE  TO  DESIGN  OF  DOO  SYSTEMS  IN  THE  SHUTUE 
ERA  AS  THE  MEANS  OF  ASSURING  MISSION  SUCCESS 

•  NASA  HAS  SUPPORTED  EXTENSIVE  GROUND  TEST  PROGRAMS  IN  AN  EFFORT  TO 

ASSURE  ADEQUATE  DEFINITION  OF  THE  ENVIRONMENT  PRIOR  TO  ACTUAL 
FLIGHT  TEST 

•  SPECIFIC  STRUCTURAL  DESIGN/TEST  CRITERIA  OPTIONS  HAVE  BEEN  DEFINED  FOR 

DOD  SPACECRAFT  TO  ALLOW  FLEXIBILITY  IN  SPACECRAFT  DESIGN  AND  TO 
FACILITATE  PROGRAM  PLANNING 

•  MAJOR  PROBLEM  AREA  HAS  BEEN  A  RESULT  OF  SPACECRAFT  AND  BOOSTER  PARALLEL 

DEVELOPMENT 

•  ACCENTUATES  VARIABILITY  IN  THE  DEFINITION  OF  THE  LOW  FREQUENCY  DYNAMIC 

LOADS  ENVIRONMENT 

•  PRECLUDES  ADEQUATE  OEFINI1ION  OF  ACOUSTIC  ENVIRONMENT 

•  ASSESSMENTS  HAMPERED  BY  UCK  OF  ADEQUATE  VISIBILITY  INTO  SUCH  AREAS 

OF  POTENTIAL  VARIABILITY  AS  SHUTTLE  LOADS  METHODOLOGY  AND  FORCING 
FUNCTIONS 


Environment  Verification  Program 

•  EXTENSIVE  GROUND  TEST  PROGRAMS 

•  SHUHLE  DYNAMIC  MODEL  VERIFICATION  PROGRAM 

•  QUARTER 'SCALE  MODEL  MODAL  TESTING 

•  HORIZONTAL  GROUND  VIBRATION  TEST  (HGVT)  AND  MATED  VERTICAL  GROUND 

VIBRATION  TEST  IMVGVT) 

•  QUARTER-SCALE  PAYLOAD  MODEL  MODAL  TESTING  WITH  SIMULATED  CARGO 

ELEMENT  MODELS 

f  STATIC  INFLUENCE  COEFFICIENT  TESTS 

•  lUS  DYNAMIC  MODEL  VERIFICATION  PROGRAM 

•  DEVELOPMENT  MODEL  MODAL  TESTING 

•  FLIGHT  CONFIGURATION  MODAL  SURVEY 
t  SPACECRAFT  DYNAMIC  MODEL  VERIFICATION 

•  MODAL  SURVEY  OR  REDUCED  LEVEL  TESTING  FOR  DYNAMICALLY  SIMPLE 

SYSTEMS 

•  SHUniE  FORCING  FUNCTIONS 

•  SRM  AND  SSME  GROUND  FIRINGS 

•  SCALE  MODEL  TESTING  FOR  DEFINITION  OF  OVERPRESSURE  FORCING  FUNCTION 


Concluding  Comments  (Cont'd) 


•  SOME  SHUTTLE-UNIQUE  TECHNICAL  PROBLEMS  HAVE  ALSO  BEEN  INTRODUUD 

•  APPARENT  NEED  FOR  EXPANDED  FRACTURE  CONTROL  REQUIREMENTS  ICURRENRY 

UNDER  REVIEW  AND  ASSESSMENT  OF  IMPACT) 

•  CARGO  BAY  ACOUSTIC  ENVIRONMENT  LEVELS  IN  LOW  FREQUENCY  REGIME  ARE 

APPARENTLY  SENSITIVE  TO  CARGO  PRESENCE.  WITH  THE  POTENTIAL  FOR 
INTERACTING  WITH  THE  LOW  FREQUENCY  STRUCTURAL  DYNAMICS 

•  FRICTION  EFFECTS  AT  CARGO  aEMENT  /ORBITtR  INTERFACES  ARE  IMPACTING 

ANALYSIS  METHOOaCY  AND  GROUND  TESTING 

#  SOME  SYSTEM  PROBLEMS  TYPICAL  Of  EXPfWMBLE  BOOSTERS  AND  AfnCDNO  STRUCT¬ 

URAL  DESIGN  PERSIST 

•  SHUTUE/CARGO  ELEMENT  FLIGHT  CONTROL  SYSTEM/ STRUCTURAL  DYNAMIC  INTER¬ 

ACTION  PROBLEM  HUS  LOW  RESPONSE  SYSTEM) 

•  DIFFICULTY  IN  OBTAINING  ADEQUATE  PROGRAM  OF  MEASURED  DATA  TO  VERIFY 

THE  ENVIRONMENTS  AND  LOADS  PREDICTION  WTHODaOGY 


•  MEASUREMENT  PROGRAM  fOR  FLIGHT 

•  OBJECT!  V£S 

•  VERIFICATION  OF  (low  frequency!  STRUCTURAL  LOADS  PREDICTION 
METHODOLOGY 

.  VERIFICATION  OF  ACOUSTIC  ENVIRONMENT 
.  DIAGNOSTICS 

•  CURRENT  LEVa  OF  EFFORT 

.  BASIC  SHUTTLE  MEASUREMENT  PROGRAM 
.  MAOS  (MODULAR  AUXILIARY  DATA  SYSTEM! 

•  CONCERNS 

.  LIMITED  MEASUREMENTS 

.  INAPPROPRIATE  FREQUENCY  RANGES  12-SO  Hz  n  O-SO  Hzl 
.  NO  CLEAR  PROGRAM  OF  DATA  ANALYSIS  AND  EVALUATION 
DEFINED 

.  NO  ON-GOING  MINIMUM  LEVa  PROGRAM  OfflNED  FOR 
DIAGNOSTIC  PURPOSES 


REUSABILITY 
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REUSABILITY 

With  the  beginning  of  the  Shuttle  era,  some 
new  issues  are  being  faced  in  space  tech¬ 
nology  which  are  old  hat  to  our  aircraft 
counterparts.  The  capabilities  of  Shuttle 
now  offer  an  opportunity  for  the  return 
and  subsequent  reuse  of  payloads  for  multi - 
missions.  In  addition,  for  Shuttle 
delivery  missions,  improving  upper  stages 
such  as  the  inertial  upper  stage  and  pay- 
load  aft  module,  new  equipment,  called 
airborne  support  equipment  (ASE),  is 
returned  with  the  Shuttle  for  subsequent 
reuse.  Equipment  in  this  category  involves 
control  systems  for  the  aft  flight  deck 
(AFD)  of  the  Shuttle,  as  well  as  structural 
support  cradles  and  interfacing  electronics 
which  remain  in  the  payload  bay  after  the 
upper  stage  has  been  deployed.  The  reuse 
of  the  equipment  introduces  a  new  dimension 
to  space  technology  in  fatigue  lift  -  both 
acoustic  fatigue  and  aircraH-type  fatigue 
considerations  with  respect  to  fracture 
mechanics.  The  Shuttle  and  Spacelab 
programs  have  instituted  fatigue  life 
analysis  and  testing  for  these  disciplines 
which  are  similar  to  what  is  deployed  in 
the  aviation  industry  today.  It  has  not 
appears  as  though  MIL  STD  1540,  the  test¬ 
ing  standard  for  spacecraft  systems,  has 
not  as  yet  been  revised  to  encompass  these 
new  dimensions,  and  certainly  these  consi¬ 
derations  are  fundamentally  a  part  of  mis¬ 
sion  assurance  in  the  Shuttle  era. 


•SHUTTLE  CAPABILITY  TO  RETURN  PAYLOADS  INTRODUCES  NEW 
ISSUES  TO  SPACE  TECHNOLOGY 


•SHUTTLE  SORTIE  MODE  i  RETURNED  AIRBORNE  SUPPORT  EQUIPItNT 
(ASE)  DICTATE  NEW  CONSIDERATIONS  FOR  FATIGUE  LIFE 

•  ACOUSTICS 

•FRACTURE  MECHANICS 


•SHUTTLE  «  SPACELAB  HAVE  IMPLEMENTED  AIRCRAFT-TYPE  OF 
TESTING/ANALYSIS  APPROACHES 


•MIL  STANDARD  1540  SHOULD  REFLECT  THIS  CHANGE  IN  SPACE 
UTILIZATION 


ORBITtR  COOftOINAT!  SYSTEM 


LIFTOFF  AND  LANDING  LOAD  ENVIRONMENT 

The  Spacelab  long  module  plus  single  pallet 
is  typical  of  how  Shuttle  payloads  have 
evolved.  Important  parameters  affecting 
the  liftoff  loads  are  the  updates  to  the 
forcing  functions  that  resulted  from  the 
SRB  DM  (development  motor),  the  QM  (qual¬ 
ification  motor)  test  firing  data,  vehicle 
modle  updates,  and  the  main  engine  pro¬ 
pulsion  test  (MPT).  Important  parameters 
affecting  the  landing  forcing  functions 
are  the  main  landing  gear  forces  that  were 
updated  after  the  approach  nd  landing 
test  (ALT).  The  maximum  loads  are  two 
sigma  values  with  a  probability  of  0.98. 
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liftoff  ahd  landing  load  environnent 

SPACELAB  LONG  NOODLE  PLUS  SINGLE  PALLET 


Lirrorr 


A. 2  PAYLOAD  BAY  ACOUSTIC  ENVIRONMENT 

The  acoustic  environment  in  the  empty 
orbiter  cargo  bay  is  produced  by  rocket 
engine  exhaust  noise  during  liftoff  and 
by  boundary-layer  pressure  fluctuations 
(aeronoise)  during  the  Mach  1,  maximum- 
dynamic-pressure  portion  of  ascent  aero¬ 
dynamic  flight. 

The  NASA  acoustic  environment  for  an  empty 
cargo  bay,  as  specified  in  NASA/JSC  07700, 
Volume  XIV,  is  shown  along  with  the  pre¬ 
dicted  environment  based  upon  6.4  percent 
model  tests.  It  es  expected,  based  upon 
previous  program  experience,  that  the  pre¬ 
dicted  values  will  shift  one  octave  to  the 
right,  and  thus,  closely  match  the  speci¬ 
fied  Volume  XIV  data.  Acoustic  level  mea¬ 
surements  collected  during  the  OFT  program 
(STS  flights  1  through  4)  will  resolve  the 
scaling  question.  Also  shown  in  the  draw¬ 
ing  is  the  Air  Force  design-to-acoustic 
environment  which  conservatively  encompas¬ 
ses  the  predicted  environment. 

PAYLOAD  BAY  ACOUSTIC  ENVIRONMENT 


LinofF 
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ISSUE;  NORST  CASE  PREDICTIONS  INDICATE  A  SLICNT  EXCEEDING  OF  SPECIFICATIONS 
IN  LONER  FREQUENCY  RANGE 

•  ON  PREVIOUS  PROGRAK.  MODEL  EXTERNAL  NOISE  DATA  PEAKS  AT 
FREQUENCIES  OVER  ONE  OCTAVE  LONER  THAN  FULL  SCALE  DATA 
•  OFT  PROGRAM  (STS  FLIGHTS  1  THROUGH  A)  NIU  RESOLVE 
SCALING  QUESTION 


A. 3  PREDICTED  RANDOM  VIBRATION  AT  UNLOADED 


The  Space  Shuttle  vehicle  will  be  subjected 
to  fluctuating  pressure  loading  on  its 
exterior  surfaces  caused  by  engine  exhaust¬ 
generated  acoustic  noise  during  liftoff 
and  aerodynamic  noise  during  atmospheric 
flight.  These  fluctuating  pressure  loads 
are  the  principal  sources  of  structural 
vibration. 

The  predicted  maximum  random  vibration  at 
the  unloaded  (i.e.  empty  cargo  bay)  long¬ 
eron  and  keel  cargo  attach  fittings, 
caused  by  these  fluctuating  pressure  loads, 
is  shown  in  the  accompanying  drawing. 

Actual  vibration  input  to  cargo  elements 
will  depend  upon  the  cargo  element  weight, 
center  of  gravity  location,  and  stiffness 
of  each  cargo  element  and  the  cargo  element 
support  and  retention  structure. 

There  is  an  estimated  95  percent  assurance, 
based  upon  solid  and  liquid  engine  test 
data  and  detailed  analysis,  that  the  pre¬ 
dicated  levels  are  accurate.  Final 
verification  will  be  obtained  during  the 
OFT  program. 


PREDICTED  RANDOM 

VIBRATION  AT  UNLOADED  LONGERON  AND  KEEL 
ATTACHMENT  FITTINGS 


PREDICTED  THERMAL  ENVIRONMENT 

The  current  predicted  thermal  environment 
in  the  cargo  bay  is  expected  to  remain  the 
same  and  will  be  updated  or  modified  as 
required  on  the  basis  of  flight  data 
obtained  during  the  initial  orb iter  flights. 

Current  predictions  of  the  thermal  environ¬ 
ment  for  various  mission  phases  are  derived 
from  analytical  results  utilizing  sophis¬ 
ticated  computer  programs  and  thermal  math 
models  of  the  orbiter  and  payloads.  A 
typical  cargo  bay  wall  temperature  range, 
as  given  in  NASA  ICD  2-19001,  "Shuttle 
Orbiter/Cargo  Standard  Interfaces,"  is 
illustrated  for  the  various  mission  phases. 

Predictions  for  specific  payloads  are 
dependent  upon  flight  parameters  and  cargo 
element  configurations,  and  thus,  require 
integrated  cargo/orbiter  analysis.  This 
is  particularly  important  if  solar  entrap¬ 
ment  could  occur  during  on-orbit  operation 
when  the  cargo  bay  doors  are  open.  Solar 
entrapment  may  occur  when  solar  radiant 
energy  is  trapped  within  a  cavity  such  as 
a  narrow  gap  or  space  between  a  payload 
surface  and  the  cargo  bay  liner.  If  solar 
entrapment  does  occur,  the  resultant 
temperature  within  the  cavity  could  reach 
high  values  and  may  exceed  temperature 
limits  of  both  the  payload  and  the  orbiter. 
Currently,  NASA  and  Rockwell  are  studying 
solar  entrapment  to  aid  in  predicting  and 
in  preventing  or  mitigating  its  effects. 

To  establish  the  on-orbit  thermal  environ¬ 
ment,  an  important  consideration  is  the 
orbiter  attitude  hold  constraints  which 
are  dependent  upon  the  orbiter  attitude 
and  angle,  as  illustrated.  There  is  a 
±30°F  uncertainty  on  range  boundaries. 

PREDICTED  THERMAL  ENVIROWrtNTS  STATUS 
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PREDICTED  CONTAMINATION  LEVELS  ALONG 
LOS  1  DUE  TO  ORBITER  SOURCES 

The  major  contamination  sources  attribut¬ 
able  to  the  orbiter  during  on-orbit 
operations  are  shown  in  the  accompanying 
drawing.  The  values  have  been  established 
by  using  the  NASA-funded  Shuttle/Payload 
Contamination  Environment  (SPACE)  computer 
program.  The  drawing  indicates  the  levels 
caused  by  various  effluents  which  a  sensor 
would  see  along  LOS  1.  The  number  column 
densities  (NCD)  give  the  number  of 
molecules  per  square  centimeter  in  the 
optical  path.  This  is  a  measure  of  the 
attenuation  which  can  be  expected  by  the 
various  contaminant  molecular  species  due 
to  absorption  and  scattering  of  the  signal 
photons.  The  return  flux  (RF)  is  a 
measure  of  the  time  rate  of  incidence  of 
the  molecular  species  upon  a  sensor 
located  at  the  LOS  1  origin. 

During  on-orbit  data  accumulation  opera¬ 
tions,  the  main  RCS  jets  are  inhibited 
and  only  the  vernier  jets  are  used  for 
station  keeping.  Two  vernier  jets  are 
located  in  the  nose,  one  on  either  side, 
and  fire  downward;  four  verniers  are 
located  in  the  aft  end,  two  on  either  side 
as  shown  in  the  drawing.  None  of  the 
verniers  fires  toward  the  region  above 
the  payload  bay. 

Flash  evaporator  ports  are  located  on 
either  side  of  the  aft  end  of  the  orbiter 
(see  drawing)  and  vent  in  the  plus  and 
minus  y  directions  simultaneously.  The 
evaporators  boil  excess  potable  water, 
generated  by  the  fuel  cells,  to  maintain 
the  subsystems  within  tolerable  temper¬ 
ature  ranges.  The  quantity  of  water 
expelled  is  a  function  of  several 
variables,  of  which  e-angle  and  power 
consumption  are  dominant.  Under  certain 
favorable  conditions  the  flash  evaporator 
system  can  be  inhibited,  thereby  removing 
a  significant  contribution  to  the  on-orbit 
contamination  environment. 

Environment  predications  similar  to  those 
shown  on  the  drawing  have  been  made  for 
several  other  LOS.  These  are  documented 
in  the  Stand  Interface  ICD  2-19001. 
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ENVIRONMENT  MEASUREMENT  PROGRAMS 

The  ALT,  SRB,  quarter  scale,  and  MPTA  test 
programs  all  are  limited  in  that  they  do 
not  reflect  the  flight  configuration  and 
launch  facilities  of  the  Shuttle.  The 
interaction  of  the  various  Shuttle  elements 
(MPS,  SRB,  ET,  launch  facilities,  and  pay- 
loads)  is  expected  to  have  a  significant 
effect  on  the  various  induced  environments 
under  consideration.  The  OFT  program  will 
be  the  first  chance  to  obtain  data  from 
all  inputs  simultaneously.  While  the  past 
and  current  programs  provided  some  confi¬ 
dence  that  the  predicted  environmental 
data  are  valid,  only  the  OFT  flights  will 
verify  the  predictions  or  provide  data  to 
update  them. 

In-flight  verification  of  payload  responses 
will  be  provided  by  38  wideband  instruments 
on  STS-2,  -3,  and  -4.  Acceleration  load 
factors  will  be  measured  by  16  low-fre¬ 
quency  accelerometers,  vibration  responses 
by  8  high-frequency  accelerometers,  and 
acoustics  by  14  microphones. 

Temperature  sensors  on  STS-3  will  measure 
payload  retention  fitting  temperatures 
during  all  mission  phases.  Frictional 
loads  at  the  fittings  will  be  determined 
from  the  temperature  data. 

The  induced  environment  control  monitor 
(lECM)  will  measure  Shuttle-  and  payload- 
induced  contamination  of  STS-1  through 
STS-4. 


ACOUSTIC 

•MPTA  TESTS 

•SRB  KVELOPKNT  A  lJUAL  TESTS* 

•QUARTER-SCALE  GVT  (INFLUENCE  OF  PAYLOADS  IN  BAY  3  CONFIGURATION)* 

•  OFT  PROGRAM 
•MADS  -  STS-2** 

LOADS 

•MPTA  TESTS  (LIFTOFF  FORCING  FUNCTIONS) 

•APPROACH  S  LANDING  TEST  (LANDING  FORCING  FUNCTIONS)* 

•GROUND  STRUCTURAL  TESTS  (SHUTTLE  MODELS)* 

•QUARTER-SCALE  GVT  (PAYLOAD-SHUTTLE  RESPONSE)* 

•  on  PROGRAM 
•MAOS  -  STS-2** 

CONTAMINATION 

•  lECM  (OFT) 

THERKAL 

•OFT  PROGRAM 

VIBRATION 

•MPTA  TESTS 
•QUARTER-SCALE  GVT* 

•OFT  PROGRAM 
•SRB* 


•COMPLETE 

**MODULAR  AUXILIARY  DATA  SYSTEM  (MADS) 

•STS-2  S  SUBSEQUENT  FLIGHTS  (TO  BE  DEFINED) 


UPDATE  FORECAST  FOR  ENVIRONMENTS 

The  Shuttle  environments  will  be  periodi¬ 
cally  updated  prior  to  Shuttle  operational 
f 1 i ghts . 

Development  and  qualification  motor  tests 
of  the  SRB's  have  been  completed.  The  re¬ 
sults,  along  with  MPT  test  results,  have 
been  incorporated  into  the  all-year,  all- 
vehicle  liftoff  forcing  functions.  Future 
liftoff  forcing  function  updates  utilizing 
flight  data  are  anticipated  during  the  OFT 
program. 

Landing  forcing  functions,  with  a  9.6-foot 
per  second  sink  rate,  are  causing  structur¬ 
al  load  problems  for  the  orbiter.  To  al¬ 
leviate  gear  loads,  the  main  gear  (metering 
pin)  may  be  modified  prior  to  operational 
flight.  This  will  necessitate  a  qualifi¬ 
cation  test  program  and  landing  gear  forc¬ 
ing  function  update. 

The  Shuttle  math  models  will  be  updated  to 
the  GVT  and  STA  test  results  for  the  OFT 
program,  and  for  the  operational  flights 
of  Orbiter  102.  Additional  model  updates 
will  be  made  for  the  lightweight  ET  and  for 
Orbiters  099  and  102. 


» 


t 


82 


F 


!• 


MULTIPLE  CARGO  ANALYSIS  STRATEGY  STRUCTURAL 
LOADS 


Preliminary  cargo  element  design  is  partly 
guided  by  the  limit  load  factors  presented 
in  JSC  0700,  Volume  XIV,  for  liftoff  and 
landing.  However,  dynamic  response  of  the 
cargo  element  is  dependent  upon  and  can  be 
magnified  by  its  location  in  the  cargo  bay 
and  the  presence  of  companion  cargo  ele¬ 
ments.  Therefore,  when  uncertainty  exists 
as  to  the  planned  location  of  a  particular 
cargo  element  and/or  the  mix  of  companion 
cargo  elements,  the  dynamic  loads  analysis 
must  be  conducted  in  a  manner  which  will 


ensure  the  design  adequacy  of  that  cargo 
element  regardless  of  its  location  or  the 
presense  of  other  cargo  elements. 


As  illustrated,  this  goal  can  be  achieved 
by  performing  the  coupled  orbi ter/cargo 
element  dynamic  loads  analysis  with  some 
multiple  of  the  cargo  element,  as  con¬ 
strained  by  cargo  element  size,  weight, 
and  the  allowable  cargo  ^  envelope 

installed  in  the  cargo  bay.  When  mixed 
with  other  cargo  elements,  the  analysis 
results  will  define  the  dynamic  loads  at 
each  of  the  potential  cargo  element  loca¬ 
tions  and  identify  the  maximum  expected 
dynamic  loads. 


Although  the  cargo  bay  acoustic  environ¬ 
ment  may  be  attenuated  by  the  presence  of 
other  cargo  elements,  design  of  a  particu¬ 
lar  cargo  element  should  anticipate  ex¬ 
posure  to  the  limit  acoustic  environment 
previously  shown.  Similarly,  the  presence 
of  other  cargo  elements  in  the  cargo  bay 
will  attenuate,  dependent  upon  their 
weights,  the  random  vibration  input  at  the 
longeron  and  keel  attach  fittings.  How¬ 
ever,  design  of  a  particular  cargo  element 
should  consider  only  that  attenuation  re¬ 
sulting  from  its  own  weight,  as  illustra¬ 
ted. 


MULTIPLE  PAYLOAD  CARGO  ANALYSIS  STRATEGY 
THERMAL  INTEGRATION 

For  each  cargo  element,  three  analysis 
cycles  will  be  performed,  starting  with 
the  payload  design  analysis  and  followed 
by  the  compatibility  analysis  and  then  the 
verification  analysis,  as  illustrated. 

As  progress  is  made  from  one  analysis  cycle 
to  the  next,  better  and  detailed  definition 
of  the  various  cargo  elements,  their  loc¬ 
ation  and  position  in  the  cargo  bay,  and 
the  mission  parameters  are>  established 
and  used  in  the  analysis. 

For  each  payload  or  cargo  element,  the 
design  analysis  can  be  performed  by 
utilizing  a  simple  cargo  bay  thermal  math 
model  with  only  the  payload  model  in  an 
assumed  location.  This  results  in  the 
worst  design  condition.  Generally,  entry 
and  on-orbit  mission  phases  result  in  a 
maximum  or  minimum  thermal  environment. 

For  payloads  which  may  be  sensitive  to 
thermal  interactions  with ‘Other  cargo 
elements,  simple  representative  thermal 
models  of  those  cargo  elements  could  be 
included. 

To  perform  the  compatibility  analysis, 
the  390-node  orbiter  thermal  math  model 
developed  by  NASA  will  be  used.  If  all 
the  cargo  TMM's  are  not  available,  repre¬ 
sentative  TMM's  would  be  used  unless 
there  is  indication  that  this  would  com¬ 
promise  the  analysis  results.  The 
objective  of  this  analysis  is  to  determine 
if  there  are  any  thermal  problems  or  issues 
associated  with  a  payload  flying  with  the 
orbiter  for  various  mission  phases. 

The  last  analysis,  the  verification 
analysis,  will  be  performed  14  months 
prior  to  launch.  For  this  analysis,  a 
complete  cargo  TMM  will  be  used  with  each 
cargo  element  location  defined.  The 
objective  of  TMM's  analysis  is  to  verify 
thermal  compatibility  for  the  planned 
operational  conditions  and  flight  profile. 

Under  contract  from  NASA,  Rockwell  will 
perform  the  compatibility  and  verification 
analyses.  These  analyses  will  be  perform¬ 
ed  as  part  of  the  standard  services.  This 
requires  each  payload  contractor  to 
provide  thermal  math  models  in  accor¬ 
dance  with  the  published  NASA  criteria/ 
guidelines  for  math  models. 

In  addition,  optional  services  are  avail¬ 
able  to  the  payload  contractor  to  augment 
the  standard  analysis  cases,  connections. 


and  thermal  models,  Optional  services 
range  from  the  development  or  modifi¬ 
cation  of  payload  thermal  math  models  to 
a  wide  range  of  integrated  analyses. 


WLTIFa  PAYLOAD  CARGO  AMLYSIS  STRATEGY 
THERMAL  INTEGRATION 
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MULTIPLE  PAYLOAD  CARGO  ANALYSIS  STRATEGY 
THERMAL  ATTITUDES 

For  multiple  payload  cargos,  particularly 
those  that  include  deployable- type  pay- 
loads,  NASA  has  established  payload  atti¬ 
tude  hold  requirements.  Essentially, 
the  payload  is  required  to  withstand, 
direct  solar  exposure,  (the  cargo  bay 
facing  the  sun)  for  at  least  30  minutes, 
and  to  withstand  a  cold  condition  (with 
the  cargo  bay  pointed  towards  deep  space) 
for  90  minutes.  To  meet  the  attitude 
hold  requirements,  payloads  are  being 
designed  with  a  removable  thermal  shroud. 

MULTIPII  PAYLOAD  CAR(»  ANALYSIS  SlRATtGY 
•  PAYLOAD  ATTITUDE  HOLD  REQUIREMENTS 
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MEASUREMENT  PROGRAMS  FOR  THERMAL  ENVIRON¬ 
MENTS 

To  provide  thermal  data  and  to  monitor 
temperatures  in  specific  areas,  17 
permanent  temperature  sensors  are  located 
in  the  cargo  bay.  In  addition,  for  the 
initial  orbiter  flights,  16  additional 
sensors  will  be  located  in  the  cargo  bay. 
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These  sensors  plus  those  that  are  located 
throughout  the  orbiter  structure  will  be 
used  to  update  and  modify  the  predicted 
temperatures  and  the  orbiter  thermal  math 
model s . 

For  the  initial  orbiter  flights,  the 
flight  attitudes  and  timelines  have  been 
carefully  selected  to  provide  thermal 
data  that  will  establish  the  temperature 
ranges  and  the  attitude  constraints. 


ICASUREHENT  PROGRAKS  FOR  TKERnAL  CNVIROItfCNTS 
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During  the  cargo  verification  cycle, 
independent  loads  and  thermal  analyses 
will  be  performed  by  Rockwell  to  assure 
orbiter-cargo  configuration  compatibility. 
The  compatibility  assessment  report  will 
be  updated  accordingly  and  FRR  certifi¬ 
cation  sheets  and  data  packages  will  be 
prepared. 


nULTIPLi  CARGO  ANALYSIS  STRATEGY 
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MULTIPLE  CARGO  ANALYSIS  STRATEGY 

A  cargo  compatibility  assessment  analysis 
will  be  performed  regarding  loads,  thermal 
conditions,  and  contamination  based  up.n 
data  provided  by  the  individual  cargo 
contractors.  Individual  and  cumulative 
cargo  element  loads  imposed  upon  the 
orbiter  will  be  assessed  as  will  thermal 
conditions  and  contamination  interactions. 
Acoustic  levels  will  be  effected  by  the 
volume  and  absorption  of  the  combined 
cargo  configuration.  Attenuation  of  ran¬ 
dom  vibration  will  be  dependent  upon  the 
weight  of  the  various  cargo  elements.  A 
preliminary  layout  drawing  and  interface 
diagram  will  be  produced  reflecting  cargo 
element  installation  requirements,  mass 
properties,  electrical  and  fluid  services, 
cargo  element  envelope,  trummion  locations, 
and  electrical  and  fluid  interfaces  with 
all  orbiter-provided  serivces.  The  effort 
will  culminate  in  a  cargo  integration 
review  and  a  compatibility  assessment 
report. 

During  the  cargo  design  cycle,  a  prelimin¬ 
ary  cargo  design  will  be  developed  which 
will  mature  into  a  baseline  design  and  a 
design  detailing  all  cargo  elements, 
flight  kits,  required  hardware  lists,  and 
assembly  and  disassembly  technical  orders. 


'THE  lUS  PERSPECTIVE 


lUS  Vehicle  Family 


John  Eckle 
lUS  Chief  Engineer 
Boeing  Aerospace  Corporation 


lUS  VEHICLE  FAMILY 


The  lUS  system  is  composed  of  a  family  of 
vehicles  tailored  for  alternative  payloads 
and  missions  and  capable  of  being  boosted 
to  low  earth  orbit  by  either  the  space 
shuttle  orbiter  or  the  Titan  T34D  launch 
vehicle.  Common  building  blocks  (modules) 
are  designed  to  be  used  in  each  of  the 
vehicle  configurations. 

Four  solid  rocket  motor  (SRM)  combinations, 
using  only  two  different  sized  motors, 
provide  a  family  of  IDS  vehicles  which 
satisfies  the  wide  range  of  performance 
and  interfacing  requirements  for  all  DOD 
and  NASA  missions.  The  large  motor  (21,400 
pounds  of  propellant)  is  used  in  the  first 
stage  of  all  vehicle  configurations.  It 
is  also  used  as  the  second  stage  of  the 
NASA  twin-stage  vehicle.  The  small  motor 
(6000  pounds  of  propellant)  is  the  second 
stage  of  the  two-stage  configuration  and 
the  third  stage  of  the  three-stage 
spinner. 

The  lUS  vehicle  functions  include  stage 
structure,  solid  rocket  motors,  reaction 
control,  thermal  control  and  avionics. 

The  avionics  functions  include  guidance, 
navigation  and  control;  telemetry,  tracking 
and  command  communications;  instrumentation; 
data  management;  electrical  power  and  dis- 
stribution;  and  on  board  computation. 

There  is  a  high  level  of  commonality  in 
the  avionics  elements  used  in  each  of 
these  vehicle  configurations. 
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lUS  DESIGN  REFERENCE  MISSIONS 

The  IDS  system  is  designed  to  accomodate 
a  wide  range  of  mission  types.  The  two 
stage  versions  (T34D  and  STS  compatible) 
are  designed  primarily  for  the  geosynchro¬ 
nous  (circular  orbit)  but  could  be  made 
to  accomodate  a  12  hour  (elliptical) 
orbit  with  a  substantial  performance  sur¬ 
plus. 

Planetary  missions  can  be  accomodated  by 
the  twin-stage  configuration  (2  large 
solid  rocket  motors  (SRM's)  -  2-stage 
vehicle)  for  low  energy  missions  or  the 
twin-plus-spin  configuration  (3-stage 
vehicle  with  2  large  SRM's  and  a  small 
spin  stabilize  3rd  stage  SRM). 


lUS  Design  Reference  Missions 
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lUS  DESIGN  PHILOSOPHY/CONSTRAINTS 

Several  key  design  philosophies  and  pro¬ 
gram  constraints  have  provided  the  fund¬ 
amental  framework  for  the  IDS  design  and 
testing  approach.  The  application  of  these 
philosophies  into  the  design  at  an  early 
time  period  are  ultimately  responsible  for 


the  flexibility  of  the  design  to  reliably 
accomodate  the  specific  missions  to  be 
flown  and  to  achieve  low  life  cycle  costs. 

The  most  fundamental  design  philosophy 
was  in  the  decision  to  design  generic 
vehicle  configurations  as  opposed  to 
specific  vehicles  to  match  each  payload. 
This  required  that  the  characteristics 
of  all  known  payloads  and  mission  types 
be  identified  and  pulled  together  into  a 
set  of  generic  spacecraft  specifications 
and  generic  mission  specifications  so 
that  a  generic  lUS  configuration  could  be 
designed.  Specific  spacecraft  known  to 
require  unique  upper  stage  features  are 
accomodated  by  providing  "production 
options"  in  the  design  which  can  be 
ordered  at  the  time  the  spacecraft  is 
designated  for  lUS.  The  generic  lUS 
design  approach  has  resulted  in  being 
able  to  maximize  commonality  of  modules 
(particularly  SRM's  and  Avionics)  between 
lUS  vehicle  configurations. 

High  reliability  requirements  obviously 
maximizes  mission  success  probability. 

This  requireinent  has  been  a  fundamental 
tool  in  the  selection  of  the  optimum 
redundancy  for  elements  of  the  system. 

High  reliability  parts  (SAMSO-STD-73- 
2C)  take  advantage  of  past  failure 
histories  to  minimize  recurrence  of  past 
part  problems  through  added  controls, 
and  application  controls. 

Qualification  and  acceptance  testing  is 
governed  by  MIL-STD-1540A  which  imposes 
margins  on  environmental  test  levels 
to  provide  conservatism  to  offset  pos¬ 
sible  error  in  the  analytical  process  of 
deriving  the  anticipated  flight  environ¬ 
ment. 

The  parallel  design/development  cycle  for 
two  new  vehicles  (orbiter  and  lUS)  has 
introduced  the  risk  of  an  iterative 
design  process  as  the  design  process  for 
each  vehicle  evolves.  The  particular 
areas  of  concern  are:  changing  dynamic 
forcing  functions,  changing  interface 
definitions,  and  the  lack  of  real  flight 
environmental  data. 


lUS  Design  Philosophy 

•  Generic  design  with  production  options 

e  Mission  specifications  for  family  of  mission 
types 

e  Maximize  commonality  of  STS/T34D/pianetary 
applications  'joint  development 

•  Accommodate  transitioning  and  new  spacecraft 
types 

•  High  reliability:  0.96  requirement,  0.98  goal 

•  Hi-qual  E/E  parts  per  SAMSO-STD-73-2C 

•  Test  to  MIL'STD.1540A 

•  Design/development  in  parallel  with  shuttle 

lUS  SUBSYSTEM  ELEMENTS 

The  generic  two  stage  vehicle  configur¬ 
ation  for  T34D  and  STS  applications 
possesses  a  high  degree  of  commonality. 
Differences  are  principally  those  required 
to  accommodate  two  radically  different 
launch  vehicles:  structural  interface 
differences,  electrical/electronic  inter¬ 
face  differences,  and  the  need  to  provide 
a  flight  termination  (destruct)  system  on 
the  T34D. 

Production  options  are  provided  in  the 
generic  design  so  that  specific  users 
may  select  features  which  may  be  desirable 
for  their  unique  mission.  Principal 
production  options  include:  a  third 
hydrazine  tank,  a  second  Tracking  Telemetry 
and  Control  string,  added  spacecraft  power 
for  T34D,  added  DC/DC  electrical  converter 
for  space  craft  power,  extendible  exit 
cone  for  SRM-2  (T34D). 


lUS  Subsystem  Elements 
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COMMONALITY 

The  use  of  common  elements  between  the 
two  stage  vehicle  and  the  twin-stage 
and  twin-plus-spin  vehicles  is  shown  on 
the  facing  page.  The  SRM's  are  conmon 
elements  and  a  high  percentage  of  the 
avionics  elements  are  common.  Missions 
will  require  some  new  avionic  functions, 
however,  and  therefore  will  require  some 
new  avionics  units.  New  functions 
for  the  planetary  missions  include: 

(1)  The  spinning  third  stage  which 
will  require  a  spin  stage 
avionics  unit. 

(2)  The  need  to  minimize  initation 
(wobble)  of  the  spinning  third 
stage  which  will  require  a 
Nutation  Control  Unit  and  an 
additional  hydrazine  system  with 
reaction  nozzles  and  associated 
hardware. 


The  lUS  structural  components  for  two  and 
three  stage  vehicles  are  not  common  so  that 
performance  can  be  maximized  for  each. 

The  third  stage  structure  and  a  portion 
of  the  second  stage  structure  is  construc¬ 
ted  of  an  advanced  graphite  epoxy  composite 
material . 

INERTIAL  UPPER  STAGE  SPACECRAFT 

The  lUS  vehicle,  through  the  SINC  program, 
is  being  integrated  with  many  transitioning 
and  new  payloads.  The  design  approach 
followed,  through  the  evolution  of  a 
generic  vehicle  design,  has  resulted  in 
minimum  and  manageable  difficulties  in 
accomodating  this  wide  range  of  payloads 
and  missions. 


(3)  A  Spaceflight  Tracking  and  Data 
Network  compatible  Tracking 
Telemetry  and  Control  system 
which  will  require  a  Spaceflight 
Tracking  and  Data  Network 
transponder. 


Common  Equipment  on  Equipment 
Support  Section 
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COMMON  EQUIPMENT 
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Inertial  Upper  Stage  Spacecraft 
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LOADS  AND  ENVIRONMENTS 

AVAILABILITY  OF  DESIGN  REQUIREMENTS 

The  basic  structural  design  loads  have 
been  unchanged  since  March  1978.  The 
process  for  change,  if  necessary,  requires 
nearly  one  year  between  the  loads  analysis 
by  BAG,  checking  and  validating  by  SD,  PIC, 
etc.,  and  negotiating  and  approving  new 
criteria.  Similarly,  the  acoustic  and 
vibration  environments  have  been  unchanged 
since  the  developmental  test  vehicle  (DTV) 
tests.  Future  Shuttle  criteria  revisions 
might  require  another  DTV  acoustic  test. 

The  "environments"  used  in  the  thermal 
design  of  the  generic  lUS  are  readily 
available  and  include  the  natural  environ¬ 
ment,  the  induced  environment,  and  the 
mission  design  environment. 

CONSERVATISM  OF  DESIGN  REQUIREMENTS 


The  design  loads  and  environments  are 
conservative  in  that  extreme  forcing 
functions  coupled  with  an  uncertainty 
factor  of  1.5  and  applying  worst  case 
load  factors  simultaneously  result  in 
large  positive  margins  for  expected  loads. 
The  vibration  environments  are  conserva¬ 
tive  through  the  application  of  MIL-STD- 
1540A  (6  db  margin  for  qual  enveloping 
of  all  response  peaks  and  test  duration 
larger  than  the  flight  environment). 

The  mission  design  environments  include 
the  design  reference  missions  (DRM)  time¬ 
lines,  Orbi ter/ payload  orientations,  and 
missions  sequences.  The  lUS  thermal 
design  timelines  encompass  the  combination 
of  worst  case  times  between  significant 
events.  This  thermal  design  timeline 
has  no  relationship  to  reality  but  it  does 


include  all  possibilities  of  event  times. 
Also  included  in  the  thermal  design  time¬ 
lines  are  a  pre-launch  environment  and 
duration  and  an  Orbi ter  abort  sequence. 

The  design  considers  Orbi ter/ payload 
orientations  which  vary  widely  from  solar 
inertial  with  sun  directly  on  the  side  of 
the  lUS  to  solar  inertial  with  sun  on  the 
end  of  the  lUS.  the  lUS  design  also 
includes  thermal  control  surfaces  and 
materials  properties  which  have  been 
degraded. 

RISK  OF  CHANGE 

The  Orbi ter  forcing  function/model 
definition  continues  to  change  and  perturb 
the  design.  Static  load  test  may  show 
capability  less  than  the  design  require¬ 
ments  but  this  risk  appears  small. 

Some  acoustic  predictions  indicate  that  the 
low  frequency  acoustic  environment  may 
increase  and  invalidate  some  component 
qual  results.  This  acoustic  environment 
will  be  confirmed  by  the  all-up  Space 
Shuttle  Main  Engine  motor  firing  later 
this  year. 

Loads  &  Environments 
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MIL-STD-1540A  TEST  REQUIREMENTS 

The  "environments"  used  in  the  thermal 
design  of  the  generic  lUS  include  the 
natural  environment,  the  induced  environ¬ 
ment,  and  the  mission  design  environment. 
The  natural  environment  includes  the  solar 
flux,  the  earth's  albedo,  and  the  earth's 
emitted  energy.  Also  of  importance  is  the 
space  radiation  environment  and  its  effects 
upon  thermal  control  materials.  These 
natural  environments  are  defined  in  NASA 
TMX-64627. 

The  lUS  thermal  design  follows  the  guide¬ 
lines  in  MIL-STD-1540A.  The  figure  on 
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the  facing  page  illustrates  the  thermal  de¬ 
sign  margins  specified  by  the  MIL-STD.  The 
(ll”C)  between  the  design  requirement 
and  acceptance  test  levels  is  to  account 
for  the  pre-test  thermal  analysis  uncertain¬ 
ties.  The  18°F  (10°C)  between  acceptance 
and  qualification  is  to  provide  confidence 
that  the  component  will  perform  at  accept¬ 
ance  level  for  prolonged  periods  of  time 
and  to  account  for  test  uncertainties. 

Hence,  the  pre-test  thermal  design  margins 
are  20°F. 

The  I  US  thermal  design  environments  are 
very  conservative  with  large  margins.  The 
reasons  being  that  the  lUS  must  accommo¬ 
date  a  large  variety  is  not  monitored 
because  it  is  payload/mission  dependent 
and  must  be  evaluated  for  each  payload 
type. 


MIL-STD-1540A  Test  Requirements 
(Thermal) 
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IMPACT  OF  STRUCTURAL  LOADS  ON  MISSION 
ASSURANCE 


Conservatism  in  design  is  intended  to  com¬ 
pensate  for  a  lack  of  confidence  in  design 
requirements  such  as  structural  loads. 

The  amount  of  conservatism  used  for  loads 
is  a  function  of  previous  experience  with 
the  launch/landing  system  and  the  article 
being  designed. 

Initial  conservatism  in  the  two-stage 
structural  design  has  protected  the  con¬ 
figuration  from  major  redesign  due  to 
changing  shuttle  loads.  The  two-stage 
lUS  system,  except  for  orbiter  interface 
loads,  is  designed  using  static  load 
factor  combinations  which  envelope  the 
worst  case  loads  factors  derived  from 
non-time-correlated  dynamic  response 
analyses.  The  dynamic  analysis  envelope 
includes  a  1.5  uncertainty  factor  on 


dynamic  response  and  is  based  on  conserva¬ 
tive  forcing  functions.  It  is  believed  that 
this  procedures  results  in  at  least  25% 
conservatism  in  design  loads.  A  less  con¬ 
servative  approach  using  time-correlated 
loads  is  used  at  the  orbiter  interfaces. 

In  addition,  the  two-stage  Airborne  Support 
Equipment  has  been  designed  with  a  low  re¬ 
sponse  system  which  provides  a  "soft"  sus¬ 
pension.  This  provides  isolation  and  an  in¬ 
sensitivity  to  changes  in  the  orbiter  dyna¬ 
mic  forcing  functions.  The  resulting  Air¬ 
borne  Support  equipment  design  is  heavy  and 
increases  the  shuttle  cargo  weight  to  ac¬ 
complish  two-stage  missions.  The  low  re¬ 
sponse  frequencies  may  also  adversely  affect 
orbiter  flight  control  system  (FCS)  stabil¬ 
ity  margins. 

The  three-stage  lUS  is  also  being  designed 
using  static  load  factors  which  envelope  the 
worst  case  non-time-correlated  dynamic  re¬ 
sults  but  the  uncertainty  factor  has  been 
reduced  to  1.0.  The  three-stage  system  will 
use  the  entire  cargo  weight  capability  of 
the  shuttle  and  the  lower  uncertainty  fac¬ 
tor  is  used  to  reduce  I US  inert  weight  and 
increase  performance.  No  low  response  sys¬ 
tem  is  used  for  the  three-stage  in  order  to 
minimize  ASE  weight  and  hence  this  system 
may  be  sensitive  to  future  shuttle  forcing 
function  changes.  An  orbiter  flight  control 
stability  problem  may  also  exist  with  this 
cargo  since  low  frequency  modes  are  present 
with  this  maximum  cargo  weight. _ 

Impact  of  Structural  Loads  on 
Mission  Assurance 

Conservatism  in  design 

•  Compensation  Tor  changing  inputs 
Two-Stage 

•  1 .5  uncertainty  faciur  used  for  2-Slage  dynamics  analyses. 

Approximately  conservatism  in  design  loads 

•  Forcing  functions  analyzed  probably  conservative 

e  Time-correlated  dynamic  loads  envelope  only  used  at  orbiter  interface 

o  “Soft  Airborne  .Support  Equipment  suspension  sy.stem  design  used  to 
dectmple  spacecraft  response  from  changing  orbiter  dynamics 

•  Results  in  high  Airborne  Support  Equipment  weight  (shuttle  cargo 
weight! 

•  Possible  Flight  Control  System  inleraiiion 
Three-Stage 

•  I  0  uncertainty  factor 

•  No  “Soft”  suspension  system 

•  Not  as  conservative  as  2-S(age 

•  Possible  Flight  Control  System  interaction 

IMPACT  OF  ENVIRONMENTS  ON  MISSION  ASSURANCE 

The  acoustic  design  environments  are  based 
on  Titan  IIIC  data  and  are  intended  to 
envelope  the  Space  Shuttle  Vehicle  and 
T34D.  These  environments  have  not  changed 
since  1976  although  several  recent  analyses 
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indicate  that  the  low  frequency  environ¬ 
ment  should  increase  and  the  high  frequen¬ 
cy  environment  should  decrease.  If  these 
changes  were  made,  shock  mounted  equipment 
environments  would  increase  while  non¬ 
shock  mounted  equipment  environments  would 
decrease.  Since  shock  mounted  items  are 
tested  to  MIL-STD-1540A  minimum,  this 
change  should  have  no  effect. 

The  avionics  are  designed  for  environments 
based  on  a  full-scale  developmental  test 
vehicle  (DTV)  environmental  test  results 
with  the  application  of  MIL-STD-1540A  mar¬ 
gins  of  safety.  This  results  in  substan¬ 
tial  conservatism  since  the  DTV  was  more 
lightly  loaded  than  the  flight  vehicle 
design  and  the  complete  set  of  cabling  was 
not  included  in  the  test. _ 

Impact  of  Environments  on  Mission 
Assurance 

*  Factors  adversely  affecting  timeliness  of  information 

•  Shuttle  acoustic  levels  based  on  Titan  III 

•  No  changes  in  acoustic  level  since  1976 

•  Conservatism  in  design 

•  MIL-STD-1540A 

•  Sensitive  equipment  shock  mounted 

•  Results  in  higher  mounting  weight 

SAFETY  REQUIREMENTS 


The  Safety  Requirements  for  the  lUS  system 
are  contained  in  "Safety  Policy  and  Re¬ 
quirements  for  Payloads  Using  the  Space 
Transportation  System"  (NASA  1976).  The 
more  significant  requirements  are  shown 
on  the  chart  on  the  facing  page. 

Safety  Requirements 


Stf«ty  policy  and  roquiramonts  for  payloads  using  tha  spaca 
transportation  systtm 


Raquiramant 
a  2  Failura  Tolaranoa 

a  In  Flight  Crawr  Monitorlng/Safing 
a  Fraetura  Control  of  Structuras 

a  Praaura  Vassals 

a  Hazardous  Matarials  Control 

a  Pyrotaehnic  Oavicas 


Impact  on  Dasign 

a  Multipla  Inhibits  for  Catastrophic 
Evants 

a  Craw  Partal  Raquirad 
a  Primary  Struetura  and  Support 
Brackatry 

a  Safety  Factor  of  4:1  or  MIL-STO  1522 
or  Fraetura  Machanies  Program 
a  Salact  Non-toxic  Non-flammabla 
Matarials 

a  Usa  NSl-1  or  MIL-STO'1512  with 
Additional  Tasting 


DESIGN  IMPACT  FROM  SAFETY  REQUIREMENTS 


Incorporation  of  the  Structural  Test 


System  safety  requirements  into  the  design 
of  the  lUS  system  has  resulted  in  cost  and 
weight  deltas.  The  impacts  are  summarized 
on  the  facing  page.  All  of  these  require¬ 
ments  are  currently  incorporated  into  the 
design  to  the  extend  indicated. 

Design  Impact  from  Safety 
Requirements 

♦  Incraasad  Waight 

a  1.4  structural  lafaty  factor 
a  Emargancy  landing  loads 

a  4.0  safaty  factor  for  prassura  vassals  (optional  to  high  cost  analysas/tasts) 
a  Incraasad  shialding  for  ordnanca  firing  circuit 

•  Redundant  safety  critical  components 

a  Additional  inhibits  for  safety  critical  functions 

•  Increased  cost 

a  Addition  of  fracture  mechanics  control  program;  structures  ar>d  tanks 
a  Material  selection  and  testing  -Flammability/toxicity 
a  Additional  testing/qualifieation  of  pyrotechnic  devices 
a  Provisions  for  extra  vehicular  activity  compatibility 
a  Waiver  processing  for  non-compliance  with  selected  requirements 

DOD/STS  SAFETY  CERTIFICATION  PROCESS 

The  DOD/STS  Safety  Certification  process, 
as  depicted  on  this  chert,  is  set  forth 
in  SAMSO  (SD)  regulation  127-4  dated 
30  November  1978. 

A  similar  process  exists  for  NASA  and 
commercial  payloads,  as  required  by  the 
NASA  Safety  Policy  and  Requirements  for 
Payloads  using  the  Space  Transportation 
System  (STS).  Concurrence  and  approval 
of  the  NASA  Safety  Review  Team  is  provided 
directly  to  the  payload  organization  and 
documented  in  the  minutes  following  each 
Safety  Review.  The  safety  compliance 
data  package  and  signed  certification  by 
the  payload  organization,  certifying  the 
compliance  of  the  payload  with  the  Safety 
Policy  requirements,  is  provided  to  NASA 
30  days  prior  to  delivery  of  the  payload 
to  the  launch  site. 

Safety  Certification  Process 

•  POD  Payloid, 

•  4  incremental  tefety  reviews  -  DOD  sefety  review  team 

*  NASA  sefety  represented  on  DOD  teem 

•  System  progrem  director  signs  certificete  of  sefety  compliertce  besed  on 
recommendetions  of  DOC^TS  sefety  certificetion  panel 

♦  Non-DOD  Peyloeds 

e  4  incremental  sefety  reviews  •  NASA  sefety  review  team  (JSC) 

•  4  separate  ground  safety  reviews  at  KSC 
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DOD/STS  Safety  Certification 
Process 


Summary 


STRUCTURAL  DiSiGN 

ASSESSMENT 

•  STRUCTURAL  DESIGN  RFOUlREMENTS 

•  FORCING  FUNCTIONS  CONTINUING  TO  CHANCE 

•  DYNAMIC  LOAD  UNCERTAINTY  FACTOR  GREATER 

FOR  TWO  STAGE  VEHICLE  THAN  THREE  STAGE 
VEHICLE  (I  B  VS  1  ai 

1  •  PREDlCTfDCliVIROUMFNTS 

1 

•  OK  WITH  C!RCA1»7S  ACOUSTICS  DATA 

tSiTGOODi 

•  CONSERVATIVE  THERMAL  APPROACH 

•  CONSERVATIVE  TEST  APPROACH 

•  MEASUREDENVIROUMEtJTS 

'  •  OlErCR£NCESiUAFC.NASA 

1  SPECIFICATIONS 

j _ 

•  SAFE  rVREOUlREMFNTS 

•  DEPENDING  UPON  EARLY  SHUTTLE  FLIGHTS  TO 
CONFIRM  PAYLOAD  SAY  ENVIRONMENTS 

•  riAJOR  OlFFEREr^CES  IN  PARTS  SPECIFICATIONS 
ANO  TESTCRITERIA 

•  SOLID  PLAN  -  WELL  INTEGRATED  INTO  DESIGN 

•  JOINT  AF/NASA  CERTIFICATION  PROCESS 

_  _ 

1  •  'HUlTlPLE  payload  CARGO 

•  INTERACTION  WITH  ORBITER  FCS  DURING 

ENTRY  AND  DESCENT  A  CONCERN 

•  RIUSEABILiry  RfOUiRFMFNTS 

_ 

•  REUSEABLE  ELEMENTS  (AIRBORNE  SUPPORT 
EQUIPMENT)  SUBJECT  TO  RECYCLE 
REFURBISHMENT 

SUMMARY 

The  overall  approach  taken  in  design  of  lUS 
is  conservative  with  good  margin  provided 
for  through  the  use  of  uncertainty  factors 
on  loads  and  environments.  Additional 
conservatism  inherent  because  generic 
vehicle  must  be  capable  of  wide  range  of 
missions,  high  reliability  parts,  MIL-STD- 
1540A  test  approach,  and  high  level  of 
redundancy.  However,  all  of  this  conser¬ 
vatism  has  resulted  in  increased  weight 
which  has  degraded  payload  performance 
capability. 

Some  fundamental  orbiter  to  lUS  interface 
concerns  still  remain  which  dictate  that 
the  highly  conservative  approach  is 
probably  appropriate: 

Changing  orbiter  structural  models 
and  dynamic  forcing  functions 

Potential  coupling  of  lUS  system 
structural  dynamics  and  orbiter 
flight  control  system 

Confirmation  of  payload  bay  environ¬ 
ments  through  flight  test  data  anal¬ 
ysis 


DESIGNING  SPACE  TELESCOPE  FOR 
THE  SHUHLE  ERA 

Marne  L.  Mercer 
Space  Telescope  Program 
Lockheed  Missiles  &  Space  Company 
ST/STS  Interface  Manager 

DESIGNING  SPACE  TELESCOPE  FOR  THE  SHUmE 
ERA 

Ten  seconds  In  the  marriage  of  the  Space 
Telescope  to  the  shuttle  produce  major 
problems  for  the  Space  Telescope  designers. 
I  will  talk  about  these  ten  seconds,  why 
they  are  troublesome,  what  Space  Telescope 
is  doing  about  the  problems  and  what  the 
Space  Shuttle  should  do  about  them.  But 
first  a  few  words  about  the  Space  Tele¬ 
scope.  It  will  be  an  astronomical  obser¬ 
vatory  in  a  300  nautical  mile  orbit 
around  the  earth.  It  is  42  feet  long  and 
14  feet  in  diameter  and  weighs  23,000 
pounds.  It  will  see  seven  times  as  far 
as  the  best  earth  observatory.  It  has 
an  8  foot  diameter  primary  mirror  and 
five  scientific  instruments,  each  about 
the  size  of  a  telephone  booth.  The  point¬ 
ing  accuracy  requirement  is  one  hundredth 
of  an  arc  second.  In  other  words,  this 
is  a  large,  high  precision  vehicle. 


SPACE  TELESCOPE  SYSTEM 


SPACE  SHUTTLE 


SUPPORT 

SYSTEMS 

MODULE 


SCIENTIFIC 

INSTRUMENTS 


CONTROL 

CENTER 


SCIENCE  :  , 

OPERATIONS  «,,«• 
FACILITY 


And  now  for  the  troublesome  ten  seconds. 
In  December  1983,  a  test  director  in 
Florida  will  push  a  button  and  over  six 
million  pounds  of  thrust  will  lift  the 
Space  Shuttle  and  the  Space  Telescope  off 
the  pad.  This  thunderous  event  does  two 
things  to  the  Space  Telescope.  It  pro¬ 
duces  most  of  the  structural  design  loads 
and  all  of  the  vibro-acoustic  design 


environments.  Where  do  we  get  the 
design  loads?  The  Space  Telescope  to 
Space  Transportation  System  Payload 
Integration  Plan  tells  us. 


TROUBLES  FOR  SPACE  TELESCOPE  DESIGNERS 

•  STRUCTURAL  DESIGN  LOADS 

•  VIBRO-ACOUSTIC  DESIGN  ENVIRONMENTS 


EXTRACTS  TROM  PARA.  6.1  OF  STS  ST  PAYLOAD  INTEGRATION  PLAN 

•■...DESIGN  SHALL  BE  BASED  OPON  COUPLED  DYNA.MIC  AND  UUAS ISTATIC  ANALYSIS 
PERFORMED  USING  UPDATED  PAYIOAD  AND  SHUHLE  MODELS,  " 

■THE  PAYLOAD  IS  RESPONSIBLE  FOR  APPLYING  APPROPRIATE  CONSERVATISM  TC  ’HE 
LOADS  TO  ACCOUNT  FOR  ANTICIPATED  .NODE!  AND  FORCING  FUNC’ION  UPDATES. 

■■...THE  RESPONSIBILITY  FOR  PAYLOAD  CC.MPATIBILITY  WITH  THE  MNAL  FLIGHT  LOADS 
REMAINS  WITH  THE  PAYLOAD. 


There's  the  rub.  What  is  "appropriate 
conservatism"  to  "account  for  anticipated 
updates"  to  assure  compatibility  with 
the  "final  flight  loads"?  We  have  run 
three  load  cycles  (in  1977,  78  and  79) 
for  the  Space  Telescope  and  the  results 
provide  some  guidance  on  how  loads  vary 
with  different  models  and  different 
transients.  Here  are  the  results  for 
the  primary  mirror  for  the  lift-off  and 
two  landing  conditions.  The  maximum  com¬ 
puted  load  factor  for  three  orthogonal 
axes  are  plotted  for  each  load  cycle. 

The  variations  are  thus  graphically 
portrayed.  We  did  this  to  help  us 
establish  an  appropriate  "variability 
factor"  to  apply  to  computed  load  factors 
in  order  to  establish  design  load  factors. 
The  objective  of  these  design  load  fac¬ 
tors  is  that  they  be  "compatible  with 
the  final  flight  loads"  which  will  be 
computed  three  years  from  now. 
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Similar  variations  but  different  magnitude 
load  factors  were  computed  for  the 
secondary  mirror  and  scientific  instru¬ 
ments.  After  studying  our  results  and 
the  results  of  other  programs,  we 
decided  to  apply  a  variability  factor 
of  1.6  to  the  latest  (1979)  computed 
load  factors  to  establish  design  load 
factors  for  the  ST.  After  discussions 
and  independent  evaluations,  Marshall 
Space  Flight  Center  concurred  with  this 
factor. 


TRANSIENT  RESPONSE  ■  SECONDARY  MIRROR 


IWA>S(KM  KiSPOSSI  ilCONUAHY  MIMHOK  ““1 


The  other  shuttle  environment  which  is  a 
major  design  driver  is  the  vibro-acoustic 
environment  of  lift-off.  This  affects 
the  structural  design  in  terms  of  addi¬ 
tional  lift-off  loads.  Even  more  signi¬ 
ficant,  it  is  a  primary  design  and  test 
environment  for  all  spacecraft  components 
For  these  reasons,  early  in  the  program, 
Lockheed  ran  an  acoustic  test  of  a  full 
scale  dynamic  test  vehicle  with  simulated 
components.  This  is  the  test  set-up. 

The  vehicle  was  supported  by  a  low 
frequency  suspension  system  and  subjected 
to  the  acoustic  spectrum  specified  in  the 
ICD.  It  was  also  subjected  to  random 
vibration  inputs  at  the  trunnions.  These 
contributed  very  little  to  the  total 
random  response  of  the  ST  elements. 


I  Of  F 


b  I  PS  LANOISC 


■J.b  ff^ 
LANOINC 


VIBRO-ACOUSTIC  TEST  SET-UP 


The  test  provided  design  acoustic  and 
random  vibration  levels  fo>"  the  various 
zones  of  the  ST  vehicle.  This  chart 
shows  the  applied  spectrum  (145  db 
overall)  and  the  various  internal  acoustic 
levels.  Not  shown  are  the  random  vibra¬ 
tion  design  levels  for  eight  different 
zones.  The  final  pre-flight  verification 
of  these  environments  will  be  an  acoustic 
test  of  the  flight  vehicle  to  maximum 
flight  acoustic  levels.  Prior  to  this, 
we  presume  that  the  Space  Shuttle  will 
have  comprehensive  verification  of  cargo 
bay  acoustics  from  early  flight 
instrumentation. 

The  final  pre-flight  verification  of  the 
structural  design  loads  from  the  ST  side 
will  be  an  ST/shuttle  coupled  analysis 
using  an  ST  model  which  has  been  verified 
by  model  tests  of  the  ST  flight  vehicle. 
Prior  to  this,  we  presume  that  Space 
Shuttle  will  have  comprehensive  verifi¬ 
cation  of  the  shuttle  model  and  forcing 
functions  from  ground  and  early  fl ight 
instrumentation. 


FREQUENCY  K) 


CONCLUSIONS 

IN  DESIGNING  SPACE  TELESCOPE  FOR  THE  SHUHLE  ERA 

•  TWO -SIGNIFICANT  RISKS  TO  SUCCESS 

•  UNANTICIPATED  STRUCTURAL  LOADS 

•  UNANTICIPATED  VIBRO-ACOUSTIC  ENVIRONMENTS 

•  ELIMINATION  OF  RISKS 

•  ANALYTICAL  AND  TEST  VERIFICATION  OF  ST  AND 
SHUHLE  MODELS 

»  EARLY  FLIGHT  VERIFICATION  OF  SHUTTLE  CARGO 
BAY  ACOUSTIC  AND  LOADING  ENVIRONMENTS 
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DESIGNING  FOR  THE  SHUTTLE  ERA 
MISSION  ASSURANCE  IMPACTS  OF 
MULTI-PAYLOAD  CARGOS 

Paul  A.  Christensen 
Martin  Marietta  Corporation 


REASONS  FOR  MULTI-PAYLOAD  CARGOS 

0  PHD  35171F 

•TO  TAKE  MAXIMUM  ADVANTAGE  THE  MULTIPLE  PAYLOAD  AND  MIXED 
CARGO  CAPABILITIES  OF  THE  SPACE  SHUTTLE* 

0  COST 

0  CONSERVATION  OF  NATIONAL  RESOURCES 


ESTABLISHINO  THE  CARGO  BIX 


UTILIZATIOB  ANALYSIS 


IMPACT  OF  LATE  CARGO  SELECTION 


0  INCREASED  LAUNCH  OPPORTUNITIES  FOR  INDIVIDUAL  PAYLOADS 


0  HIGHER  RISK  OF  DESIGN  INCOMPATIBILITIES 


0  POSSIBLE  DESIGN  CONSTRAINTS  TO  ACCOMMODATE  FLEXIBILITY 

0  COSr/SCHEDULE  RISK  OF  CARGO  ELEMENTS  UNCOUPLED  DURING 
DESIGN  PHASE 


MULTI-PAYLOAD  CARGO  IMPACTS  ON  MISSION  ASSURANCE 

0  POTENTIAL  COMPROMISE  OF  DESIGN  AND  TEST  MARGINS  TO  ACCOMMODATE 
CARGO  COMPLEMENT 

0  DESIGN  MAY  COi'IPROMISE  MISSION  CAPABILITY 

-  POTENTIAL  MIXERS  MUST  BE  INSENSITIVE  TO  ENVIRONMENT 

-  POTENTIAL  MIXERS  MUST  MINIMIZE  IMPACT  ON  ENVIRONMENT 

0  SCHEDULE  OF  OTHER  CARGO  ELEMENT  HAY  COMPROMISE  NORMAL  PREUUNCH 
SEQUENCE 

0  FAILURE  OF  ONE  CARGO  ELEMENT  MAY  IMPACT  MISSION  OF  ACCOMPANYING 
ELEMENTS 

0  ADDITIONAL  INTERFACES  PROVIDES  ADDITIONAL  RISK 


IMPACT  OF  EARLY  MIX  SELECTION 

0  CARGO  DESIGN  PROCEEDS  IN  PARALLEL  WITH  S/C  DESIGN  AND  CARGO 
ELEMENT  INTEGRATION 

0  DISCIPLINED  CONTROL  OF  COMPATIBILITY  AMONG  CARGO  ELEMENTS 

0  INTERACTIVE  COST/SCHEDULE  RISK  AMONG  CARGO  ELEMENTS 


0  LIMITED  LAUNCH  OPPORTUNITIES 


PAUL  DICK 

GENERAL  ELECTRIC  COMPANY  SPACE  DIVISION 


Ladles  and  Gentlemen,  or  shall  I  say 
persons?  Welcome  to  the  1980  Mission 
Assurance  Conference  Session  "Effective 
Failure  Analysis  In  the  Development 
Cycle".  I  am  your  Session  Chairman. 

My  affiliations  are  Paul  Dick,  System 
Effectiveness  Manager,  General  Electric 
Company  Space  Division. 

Before  I  introduce  the  distinguished 
four  speakers,  let  me  tell  you  why  we  are 
doing  this  program  for  you.  The  value  of 
failure  analysis  in  the  development  cycle 
of  a  high  reliability  product  is  not 
challenged  by  anyone  other  than  an  oc¬ 
casional  overspent,  very  late  in  delivery 
program  management  type  who  has  many 
serious  worries  on  his  mind.  The 
majority  of  the  population  recognizes  its 
value  and  benefit  to  product  reliability 
and  long  life. 

When  we  enter  into  deeper  discussions 
on  the  subject  of  failure  analysis, 
several  key  administrative,  operational, 
application,  authority,  responsibility, 
and  customer  requirements  enter  into  the 
picture.  Some  of  these  popular  issues  or 
questions  include: 

•  Where  should  responsibility  lie  for 
Failure  Analysis  in  a  project 
organization? 

•  What  is  and  should  be  the  role  of 
the  customer  and  his  participation 
in  the  failure  analysis  process? 

•  What  is  the  value  of  failure  analysis 
to  a  program's  success  and  schedule? 

•  Are  there  ways  to  profit  from  the 
results  of  a  failure  analysis  on 
current  and  future  programs? 

•  How  effective  is  failure  analysis  to 
assure  lasting  corrective  action? 

•  What  types  of  methods  can  be  used  for 
exchanging  problem  and  corrective 
action  information  interchanges 
between  various  organizations  on 
items  of  common  interest? 

•  How  adequate  are  customer  specifi¬ 
cations,  requirements,  and  work 
statements  on  failure  analysis 
(including  MIL-Specs,  NASA-Specs, 
etc.  where  applicable)? 


•  What  recommended  improvements  are 
advised? 

•  Which  are  the  practical  methods  of 
performing  FMEA? 

•  What  is  and  should  be  the  degree  of 
designer  involvement  in  FMEA? 

•  How  can  sneak  circuit  analysis  be 
used  effectively? 

•  How  does  one  assure  effective  manage¬ 
ment  of  a  single  point  failure  and 
control  of  critical  characteristics? 

I've  asked  each  of  the  speakers  that  I 
will  soon  introduce  to  try  and  cover  a  few 
of  these  points  in  his  presentation 
materials  which  I  am  sure  you  will  enjoy 
and  find  interesting.  Our  speakers,  in 
advance  of  their  individual  introductions, 
are  two  user  types  and  two  customer  sup¬ 
port  types  alphabetically  as  follows: 

Mr.  Edgar  Doyle,  Manager 
Quick  Reaction  Failure  Analysis 
RADC 

Mr .  Ronald  Hanni 
Senior  Reliability  Engineer 
Ford  Aerospace  &  Communications  Cotp. 

Mr.  Joseph  R.  Howell 
System  Effectiveness  Engineer 
Aerospace  Corporation 

Mr.  Rick  Simon 
Process  Control  Manager 
General  Electric  Space  Division 

To  keep  it  fair  geographically  wise 
after  I  get  off  the  stage,  I've  set  the 
rotation  as  follows; 

1.  An  East  Coast  Aerospace  Contractor's 
View  (Mr.  Simon) 

2.  A  West  Coast  Aerospace  Contractor's 
View  (Mr.  Hanni) 

3.  An  East  Coast  Customer's  Agency 
View  (Mr.  Doyle) 

A.  A  West  Coast  Customer's  Agency 
View  (Mr.  Howell) 

I'll  introduce  each  speaker  and  let  him 
do  his  thing  for  20-25  minutes  each.  I  ask 
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that  you  keep  your  questions  until  after 
the  last  speaker  Is  completed  and  I  turn 
the  discussion  over  to  the  audience. 
However,  to  maintain  some  rules  of  order, 
I  sincerely  ask  that  you  please  fill-in 
the  questionnaire  cards  available  in  the 
audience  and  send  them  up  to  me  as  each 
speaker  gets  finished.  I  will  collect, 
catalog,  and  ready  these  questions  for 
presentation  to  bur  panel  or  a  particular 
speaker  if  so  noted  on  the  card  for  the 
discussion  period  after  the  last 
presenter . 

I  promise  that  we  will  try  and  have 
all  questions  submitted  answered,  time 
permitting,  at  this  meeting.  Those  not 
addressed,  will  be  assigned  to  various 
panel  members  by  me  to  have  them  respond 
via  letter  to  you,  so  please  put  your 
name  and  address  on  the  questionnaire 
card  unless  it  is  so  explosive  that  you'd 
like  to  remain  anonymous. 

When  we  run  out  of  question  cards,  you 
may  ask  questions  from  the  audience.  Of 
course,  once  a  question  is  read  from  a 
card  and  answered  up  here,  you  may  want 
to  raise  your  hand  out  there  and  be 
recognized  for  further  discussion, 
whether  it  was  your  question  or  not  being 
discussed. 

This  panel  is  vitally  interested  in 
what  you  think,  need,  and  suggest,  so 
please  be  part  of  the  interchange  after 
the  last  speaker.  We  need  your  help  as 
we  the  panel  will  present  a  report  to  the 
organizing  committee  at  the  end  of  this 
conference  to  see  where  we  go  next  and 
your  comments  are  vital  so  that  it  is  not 
just  what  us  five  "Super  Stars"  think, 
need,  and  have  suggested. 

At  this  point,  I  say  thank  you  for 
your  patience  to  hearing  me  rattle  on  and 
I  with  pleasure  will  now  introduce  our 
four  distinguished  speakers. 


EFFECTIVE  FAILURE  ANALYSIS  IN  THE  DEVELOPMENT  CYCLE 


Ronald  G.  Hanni 

Ford  Aerospace  And  Cooununlcaclons  Corporation 
Western  Development  Laboratories 
Palo  Alto,  Calif. 


Introduction 

The  role  of  failure  analysis  is  to  posi¬ 
tively  impact  the  probability  of  success 
(Pg)  of  a  program  by  analyzing  failures 
and  providing  the  information  necessary 
to  take  effective  corrective  action,  re¬ 
ducing  the  likelihood  of  subsequent  simi¬ 
lar  failures.  Well  executed  failure 
analysis  will  fulfill  that  role.  This 
paper  will  outline  the  structure  of  the 
failure  analysis  effort  as  performed  on  a 
typical  program,  and  the  benefits  derived 
therefrom. 

Effort  Level  and  Benefit 
In  considering  the  structure  of  failure 
analysis  it  becomes  Important  to  realize 
that  in  effectively  managed  organizations, 
much  cross-program  impact  of  and  benefit 
from  failure  analysis  occurs  due  to  the  - 
commonalities  of  technology  being  utiliz¬ 
ed  amongst  the  various  programs.  Often 
technology  or  hardware  developed  for  one 
program  becomes  an  "off  the  shelf"  item 
for  subsequent  programs.  Hence,  effect¬ 
ive  failure  analysis  begins  prior  to  a 
contract  award  and  may  affect  more  than 
one  program.  Additionally,  development 
work  is  often  performed  during  the  course 
of  a  program  in  order  to  solve  problems 
encountered  or  to  improve  the  end  product. 

These  ideas  are  displayed  and  developed 
further  in  Figure  1  which  provides  a  rep¬ 
resentation  of  development  work  and  fail¬ 
ure  analysis  effort  vs  time  during  a  typ¬ 
ical  program.  There  is  an  underlying 
level  of  general  product  development  work 
and  associated  failure  analysis  which  is 
always  being  performed.  As  the  techno¬ 
logical  requirements  for  a  program  are 
clarified  during  the  pre-proposal  and 
proposal  efforts,  goal  orientated  develop¬ 
ment  begins,  nils  focused  development 
accelerates  greatly  at  the  award  of  the 
contract  and  during  the  prototype  phase. 

As  processes,  designs  and  hardware  mature 
through  the  production  phase,  the  develop¬ 
ment  effort  declines  to  the  background 
leve 1 . 

The  positive  Impact  of  failure  analysis 
becomes  important  in  the  early  stages  of 
a  program,  especially  in  the  proposal  and 


prototype  phases  when  new  processes  and 
components  are  being  used.  Here  failures 
can  be  analyzed  and  documented  yielding 
appropriate  corrective  actions  which  are 
instituted  to  prevent  recurrence  of  simi¬ 
lar  failures.  At  this  point  failure 
analysis  can  detect  problems  in  design, 
process,  part  application,  assembly  and 
test . 

An  example  of  effective  failure  analysis 
and  ensuing  corrective  action  occurring 
early  in  a  program  is  provided  by  lot 
oriented  migratory  gold  resistive  short 
failures.  During  prototype  and  flight 
powered  thermal  testing  of  comm  subsystem 
front  end  modules,  intermittent  base 
emitter  shorts  of  an  r.f.  transistor  were 
encountered.  After  removal,  electrical 
tests  found  the  transistors  to  be  good. 
During  powered  monitored  thermal  cycling 
and  cold  temperature  soaks,  it  was  found 
that  intermittent  base-emitter  shorts 
could  be  produced.  After  a  short  time 
these  intermittents  became  solid  failures. 
Subsequent  S.E.M.  and  X-ray  spectographlc 
analysis  of  the  transistors  identified 
the  failure  to  be  due  to  migratory  gold 
resistive  shorts  as  shown  in  Figures  2 
through  4. 

These  shorts  formed  between  the  unglassi- 
vated  gold  metallization  fingers  having 
a  spacing  of  3u  under  a  combination  of 
high  package  moisture  content,  bias  and 
time  at  cold  temperatures.  The  require¬ 
ment  of  a  high  moisture  content  in  the 
packages  and  the  occurrence  of  multiple 
failures  made  the  conclusion  that  the 
problem  was  lot  oriented  inescapable. 

Due  to  the  non-availability  of  these 
transistors,  and  the  presence  of  an  ac¬ 
ceptable  screened  substitute  transistor 
type  in  flight  stores,  replacement  was 
made  with  the  new  transistor  type.  In 
this  instance,  effective  failure  analysis 
and  appropriate  corrective  actions  avoid¬ 
ed  a  serious  program  impact  and  prevented 
repetitive  failures. 

During  the  production  cycle  of  a  program, 
failure  analysis  continues  to  play  an  im¬ 
portant  role  in  the  success  of  a  program. 
During  the  production  cycle  the  design 
has  been  proven, flight  quality  parts  are 


being  used,  and  well  controlled  assembly 
and  test  procedures  are  being  utilized. 

A  failure  at  this  point  can  represent  an 
oversight  of  design,  assembly  or  test  or 
inherent  defects  in  parts  which  have  suc¬ 
cessfully  completed  screening. 

Organization 

To  be  effective,  failure  analysis  should 
be  performed  by  a  dedicated  support  ser¬ 
vice  which  is  independent  of  design,  manu¬ 
facturing,  and  test  organizations.  This 
support  organization  should  have  full 
technical  responsibility  for  the  conduct 
of  the  failure  analysis.  When  critical 
failures  occur  with  large  Impact  poten¬ 
tials  a  "task  force"  approach  is  often 
employed  with  the  Program  Office  monitor¬ 
ing  the  analysis,  providing  managerial 
direction  and  allocating  company  resources 
as  needed.  The  Program  Office  also  pro¬ 
vides  high  level  customer  interface  in 
these  circumstances.  This  organization 
is  represented  in  Figure  5. 

I  Customer  participation  is  normally  limit¬ 

ed  to  review  of  the  results  of  the  failure 
analysis  and  corrective  actions  implement¬ 
ed.  However,  at  high  levels  of  critical¬ 
ity  and  impact  potential  the  customer  is 
routinely  involved  in  the  form  of  consult- 
.  ant  and  real  time  monitoring  functions 

'  interfacing  through  the  Program  Office. 

Customer  requirements  concerning  failure 
analysis  contain  the  requirement  that 
closed  loop  failure  analysis  be  performed 
when  failures  occur  at  or  after  the  first 
powered  module  level  test.  Additionally, 

I  failures  occurring  at  system  level  tests 

are  reportable  to  the  customer  within  24 
hours.  The  customer  is  provided  copies 
of  the  failure  analysis  reports  for  re¬ 
view. 

Failure  analysis  is  performed  to  cost  ef- 
I  fective  internal  procedures,  which  have 

been  developed  over  the  course  of  several 
programs.  These  procedures  are  general¬ 
ized  guidelines  which  require  that  experi¬ 
enced  competent  analysts  evaluate  the  data 
and  hypothesis  at  each  step  of  the  analy- 
I  sis.  These  procedures  essentially  re- 

!  quire  that  the  failure  be  confirmed,  the 

failure  Isolated,  and  the  cause  determin¬ 
ed.  The  analysis  is  fully  documented. 

Schedule  Impact 

The  Impact  of  failure  analysis  on  a  pro- 
.  gram  schedule  varies  with  the  point  in 

!  the  program  when  the  failure  occurs  and 

with  the  combination  of  the  failure  criti¬ 
cality  and  cause.  Failures  which  occur 


early  in  the  design  phase  of  a  program 
are  the  most  productive  failures  to  oc¬ 
cur.  Effective  failure  analysis  and 
corrective  action  can  prevent  recurrence 
and  prevent  schedule  impact  at  a  later 
time  when  schedule  recovery  is  more  diffl 
cult.  As  the  program  progresses,  the  im¬ 
pact  of  failure  analysis  on  a  program 
usually  becomes  more  severe  as  there  are 
fewer  opportunities  to  recover  from  the 
consequences  of  a  failure  -  especially 
where  extensive  rework  and/or  retrofit  is 
Involved. 

Due  to  a  recently  instituted  procedure, 
some  additional  positive  schedule  Impact 
occurs,  as  shown  in  Figure  6.  The  func¬ 
tional  electrical  condition  of  electron¬ 
ic  parts  suspected  of  having  caused  a 
failure  are  determined  within  24  hours 
of  receipt  by  the  failure  analysis  lab¬ 
oratory.  Meanwhile  no  rework  or  repair 
is  performed  until  it  is  determined  if 
the  suspect  part  is  good  or  bad.  If  bad, 
rework  and  repair  proceeds.  If  good, 
further  troubleshooting  is  performed  by 
MRB  direction.  This  positively  impacts 
schedule  by  preventing  rework  which  may 
not  solve  the  problem  and  which  may  need 
to  be  repeated.  This  also  prevents  de¬ 
gradation  of  hardware  by  repeated  rework/ 
repair  cycles. 

Additional  benefit  to  a  program's  success 
and  schedule  can  be  realized  by  applica¬ 
tion  of  failure  analysis  technology,  es¬ 
pecially  non- destructive  techniques,  to 
material,  part,  and  hardware  evaluation. 
Failure  analysis  can  sometimes  be  per¬ 
formed  in-situ,  gathering  enough  infor¬ 
mation  to  decide  probable  causes  of  fail¬ 
ure  and  to  whether  or  not  to  continue 
with  the 'test  which  discovered  the  fail¬ 
ure  . 

Figure  7  illustrates  an  example  where  a 
non-destructive  technique.  X-ray  radio¬ 
graphy,  provided  an  answer  as  to  the 
cause  of  a  fai  ure  occurring  during  ther¬ 
mal  cycling.  The  object  shown  in  the 
figure  is  a  quarter  inch  termination  load 
resistor  whose  bifurcated  contact  had 
been  soldered  to  a  stripline  board.  The 
board  is  part  of  an  antenna  feed  array 
consisting  of  three  stripline  layers  with 
aluminum  honeycomb  and  feedhorns.  The 
assembly  failed  in  this  one  element  (of 
approximately  40)  after  194  thermal  cy¬ 
cles  of  a  planned  test  of  400  cycles. 

One  choice  presented  was  to  halt  the  test 
disassemble  the  feedboards,  remove  the 


resistor,  perform  failure  analysis,  and 
then  re-assemble  the  feedboard  In  order 
to  resume  the  test.  Another  choice  pre¬ 
sented  was  to  X-ray  the  assembly  In  the 
failed  area  to  determine  If  the  cause 
could  be  found.  From  the  X-ray  It  was 
apparent  that  the  cause  of  the  failure 
was  not  a  lot  related  failure  of  the  re¬ 
sistor,  but  a  mechanical  overstress  which 
probably  occurred  during  assembly.  It 
was  decided  that  the  test  could  be  re¬ 
sumed  without  repair,  and  without  signifi¬ 
cant  schedule  Impact.  As  corrective  ac¬ 
tions,  all  flight  feedboard  assemblies 
were  X-rayed  to  Inspect  for  any  discrep¬ 
ant  conditions  occurring  In  the  resistor 
assemblies.  The  assembly,  test,  and  In¬ 
spection  personnel  were  cautioned  about 
careful  handling  and  Inspection  using 
this  case  as  an  example.  Here  schedule 
Improvement  resulted  from  applying  a  non¬ 
destructive  technique  to  perform  effect¬ 
ive  failure  analysis. 

Closing  the  Loop 

The  results  of  the  failure  analysis  are 
Integrated  Into  current  programs  via  re¬ 
port  dissemination  and  corrective  ac¬ 
tions.  Report  distribution  Includes  rep¬ 
resentative  of  program  offices,  customer 
organizations,  design  engineering,  line 
management  and  part  procuring  activities. 
This  is  shown  in  Figure  8.  Monthly  FRB's 
(Failure  Review  Boards)  are  convened  which 
review  problems,  the  results  of  failure 
analysis  and  the  effectiveness  of  cor¬ 
rective  actions.  This  is  in  addition  to 
the  real  time  monitoring  activities  of 
critical  failures  by  the  applicable  pro¬ 
gram  office  and  customer  representatives . 
Quarterly  reports  also  include  a  Failure 
Summary  Report.  Failure  analysis  infor¬ 
mation  is  additionally  entered  into  a 
computerized  Index  which  summarizes  the 
results  of  the  failure  analysis  and  allows 
access  by  problem  reference  numbers,  part 
numbers,  and  programs.  Cross-program  in¬ 
formation  dissemination  is  accomplished 
due  to  commonalities  of  all  involved 
parties  except  program  offices  and  cus¬ 
tomer  organizations.  Representatives  of 
all  program  offices  are  Included  on  fail¬ 
ure  analysis  report  distributions.  Ex¬ 
ternal  information  exchange  Is  accomplish¬ 
ed  In  a  real  time  basis  through  customer 
organizations,  which  frequently  monitor 
more  than  one  program  and  more  than  one 
contractor,  and  through  part  vendor  con¬ 
tacts. 


It  is  an  internal  policy  that  vendors  are 
routinely  notified  of  problems  encoun¬ 
tered  in  failure  analysis  where  vendors' 
manufacturing  processes  are  the  cause  of 
or  a  contributor  to  the  failure  of  a 
part.  This  frequently  occurs  through 
meetings  with  vendor  representatives  and 
their  technical  staff.  Occasionally, 
vendor  "education"  occurs  through  labora¬ 
tory  demonstration  of  problems  and  de¬ 
fects,  followed  by  aid  to  vendors  In 
their  product  development  efforts  and 
close  monitoring  of  vendors  manufactur¬ 
ing  lines.  This  requires  a  large  amount 
of  goodwill  between  the  vendor  and  the 
contractor . 

In  appropriate  cases  the  GIDEP  system  is 
used  to  alert  the  Industry  to  serious 
part  problems.  Symposia,  professional 
society  meetings,  and  trade  publications 
are  used  to  further  disseminate  the  In¬ 
formation.  However,  internal  budgetary 
and  procedural  constraints  limit  the  ef¬ 
fectiveness  of  these  systems. 

Program  corrective  actions  include  rework, 
retrofit,  and  evaluation  cycles,  as  well 
as  procedural  and  procurement  specifica¬ 
tion  revisions.  Lasting  corrective  ac¬ 
tion  is  organizationally  and  timewise  a 
problem  distinct  from  failure  analysis. 
Lasting  corrective  action  occurs  as  a  re¬ 
sult  of  changes  Implemented  In  design, 
manufacturing,  test  and  procurement  or¬ 
ganizations.  It  is  the  responsibility  of 
those  organizations  to  make  appropriate 
Changes  in  response  to  problems  and  to 
ensure  their  future  applicability.  The 
changes  are  subject  to  review  by  program 
offices,  customer  organizations  and  re- 
liabllity/quallty  control  organizations. 
Other  reliability  tools  such  as  FMEA 
(Failure  Modes  and  Effects  Analysis) , CARR 
(Circuit  Analysis  Reliability  Report), 
and  DPA  (Destructive  Physical  Analysis) 
systems  monitor  lasting  corrective  action 
effectiveness.  These  reliability  analy¬ 
ses  are  performed  at  various  phases  of 
each  individual  program. 

An  example  of  lasting  corrective  action 
Is  illustrated  by  a  conductive  particle 
inclusion  problem.  Loose  conductive  par¬ 
ticles  have  caused  failures  by  shorting 
internal  lead  wires  to  die  scribe  moats 
in  semiconductor  devices  with  the  fail¬ 
ures  normally  occurlng  during  vibration. 
Figure  9  shows  an  example  of  failure 
caused  by  AuSn  lid  sealing  material  while 
Figure  10  shows  a  failure  caused  by  ex- 


traneous  aluminum  slivers  generated  dur¬ 
ing  wire  bonding.  These  devices  success¬ 
fully  passed  all  the  screening  require¬ 
ments  Including  pre-cap,  X-ray,  FIND,  and 
672  hrs  of  burn-ln. 

As  a  result  of  these  failure  analyses, 
considerable  additional  DPA  effort  was 
performed  on  all  lots  received  from  this 
vendor.  Meetings  with  the  vendor  were 
held  which  involved  technical  sessions 
demonstrating  the  presence  of  included 
particles  in  their  devices,, and  techniques 
for  particle  detection  and  identification. 
After  the  vendor  accepted  that  a  problem 
existed,  large  amounts  of  engineering  ef¬ 
fort  were  expended  to  determine  the  cause 
of  the  AuSn  material  and  the  source  of 
the  A1  slivers.  Corrective  actions  in¬ 
cluded  changes  in  the  vendor's  processing 
and  inspection  procedures  and  in  the  pro¬ 
curement  specifications.  The  effective¬ 
ness  of  these  measures  was  verified  by 
analysis  of  evaluation  samples.  Concur¬ 
rently,  with  the  problem  identification 
and  correction  activities,  heightened  pre¬ 
seal  inspection  and  DPA  efforts  were  per¬ 
formed.  The  lasting  effectiveness  of 
these  changes  are  monitored  during  DPA. 

Improvements 

A  need  exists  for  greater  availability  of 
detailed  failure  analysis  procedure /tech¬ 
nique  information  and  training  sessions. 
This  may  be  alleviated  by  a  new  technical 
society  which  is  in  the  formative  stages, 
the  International  Society  for  Testing  and 
Failure  Analysis  (ISTFA) .  Targeted  con¬ 
cise  summaries  of  failure  experience  con¬ 
cerning  specific  failure  modes,  analyti¬ 
cal  techniques,  and  specific  part  or 
vendor  devices  would  also  be  helpful. 

Conclusions 

Effective  failure  analysis  will  positive¬ 
ly  impact  the  probability  of  success  of  a 
program  by  reducing  the  likelihood  of  re¬ 
petitive  similar  failures.  An  organiza¬ 
tional  structure,  providing  for  an  inde¬ 
pendent  support  service,  to  achieve  the 
goal  of  effective  failure  analysis  has 
been  illustrated.  Failure  analysis  has 
been  shown  to  be  extremely  important  dur¬ 
ing  the  development  phases  of  a  program, 
as  evidenced  by  the  level  of  effort  re¬ 
quired  and  the  positive  impact  potential. 
Effective  failure  analysis  has  been  shown 
to  result  in  schedule  improvements  in  some 
instances.  Examples  of  failure  analyses 
and  their  corrective  actions  have  been 


given.  Some  areas  of  recommended  improve 
ments  have  been  outlined. 
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Figure  3.  Arrows  indicate  location  of 
sites  of  electromigration  on 
die  pictured  in  Figure  2.  The 
metallization  fingers  are 
identified  by  function. 


Figure  10.  Arrows  indicate  aluminum 
slivers  shorting  the  Vcc 
supply  lead  to  the  die  moat 
(electrically  GND)  of  a 
quad  comparator. 
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OVERVIEW  —  SYNOPSIS  OF  PRESENTATION 

The  series  of  vu-graphs  which  com¬ 
prise  this  presentation  are  divided  into 
the  following  seven  subjects; 

•  Value  of  failure  analysis  to  a 
program's  success  and  schedule 

•  Profiting  from  failure  analysis 
resul ts 

•  Failure  analysis  effectiveness 
to  assure  lasting  corrective 
action 

•  Exchanging  problem  and  corrective 
action  information 

«  Recommended  improvements 

•  Organizational  responsibility  for 
failure  analysis 

•  Customer's  role  in  failure 
analysis 


The  following  sections  of  this  paper 
elaborate  on  each  of  the  above  svibjects. 


VALUE  OF  FAILURE  ANALYSIS  TO  A  PROGRAM'S 
SUCCESS  AND  SCHEDULE 

It  is  the  author's  belief  that  the 
success  of  a  program  is  greatly  enhanced 
bv  effective  failure  nnalvsis.  However, 
it  must  be  noted  that  a  program's  sched¬ 
ule  will  be  negatively  impacted  by 
failure  analysis  results  if  costly  and 
time-consuming,  retrofitting  and/or 
design  changes  are  necessary. 

PROFITING  FROM  FAILURE  ANALYSIS  RESUl.TS 

The  major  thrust  for  an  effective 
failure  analysis  is  to  identify  the 
cause  of  failure  so  that  appropriate 
corrective  action  can  be  identified 
and  implemented  in  order  to  preclude 
failure  recurrency.  To  this  regard, 
action  items  are  assigned  to  the 
operation  that  has  the  resources  to 
accomplish  the  necessary  effort. 


Sometimes,  this  will  include  return  of 
the  failed  item  to  a  vendor's  facility 
for  more  extensive  analysis  of  identi¬ 
fying  the  failure  mode.  When  action 
items  that  cannot  be  completed  immedi¬ 
ately  are  assigned,  an  initial  Failure 
Analysis  Report  is  published.  Results 
of  corrective  actions  are  published  in 
one  or  more  supplements. 

A  successful  method  utilized  by 
General  Electric  Space  Division  to  - 
apply  "lessons-learned"  from  the  failure 
is  to  widely  communicate  the  results 
through  extensive  distribution  of  the 
Failure  Analysis  Report.  Since  we 
utilize  a  matrix  management  concept  at 
General  Electric,  Product  Assurance 
Project  Engineers  on  all  programs  arc 
given  copies  of  all  Failure  Analysis 
Reports.  Functional  management  in 
Product  Assurance,  Manufacturing,  and 
Engineering  are  likewise  added  to  dis¬ 
tribution.  Eacli  distribution  list  is 
tailored  to  include  failure  analysis 
participants,  other  concerned  individ¬ 
uals,  and  customer/vendor  representa¬ 
tives,  as  required. 

The  Corrective  Action  Board  (CAB) 
which  meets  monthly  is  another  veliicle 
utilized  to  assure  that  failure  analysis 
results  are  appropriately  implemented. 

In  accordance  witli  M1L-STD-1520A,  senior 
functional  managers  in  Quality,  Engi¬ 
neering,  and  Manufacturing,  meet  monthlv 
to  discuss,  among  other  items,  those 
failure  analyses  that  are  of  signific¬ 
ance  to  the  Space  Division.  Customers 
and  Program  Offices  send  observers  to 
these  CAB  meetings  and  ag.ain,  the  minu¬ 
tes  are  widely  distributt'd  to  kev 
personnel . 

A  final  vehicle  to  assure  that 
failure  analysis  results  are  effect ivelv 
implemented  is  through  a  program  uninue 
Failure  Review  Board  (FRB) .  The  FRB  is 
convened  monthlv  and  consists  I'f  exten¬ 
sive  discussions  between  customer  and 
contractor  personnel  on  s i ipi i 1 i can t 
nonconformance  reports  and  on  all  pro¬ 
gram  Failure  Analysis  Reiiorts.  These 
discussions  involve  the  individual 
Quality  Control  Engineer  who  has  respon¬ 
sibility  for  the  nonconforming,  item 
under  discussion  aivi  the  Failure  Analv- 
sls  Chairman  or  Failure  Analwsis 
Engineer  who  lias  conducted  the  failuri’ 


analvsis  iindor  discussion. 

KA  II.UKF.  AXALYS  IS  KFnXTJVj-ryiSS  JO  j\SSURK 
I.ASlMNCr  CORRI-xYtvY  ACTION  ~ 

Throe  ilifferent  illustrative  exam- 
])les  are  utilixed  to  depict  tiiroe  dif¬ 
ferent  tvpi'S  of  failure  modes  associated 
with  component  piece  parts.  Of  specific 
interest  is  the  diversity  of  failure' 
analvsis  participants. 

On  oiu'  program,  an  intermittent 
logic  lircuit  was  detected  during  low 
temperaturi'  testing  of  a  complex  printed 
circuit  board.  The  suspect  failuri'  .area 
fi'cused  on  two  fl.itpack  IC's.  Solder 
joints  appeared  normal.  During  f.ailure 
t  roubl  eslioo  t  ing/ i  nves  t  iga  t  ion ,  it  w.as 
nc'ted  th.at  si  ight  pressure  on  a  diode 
loc.ated  on  the  other  side  of  the  PCB, 
exactlv  opposite  the  two  suspect  IC's, 
resulted  in  duplication  of  the  failure 
mode.  Further  .analysis  indicated  that 
a  region  of  chipped  paint  on  the  diode 
located  exactly  on  top  of  a  plated 
through  hole,  [>ermittod  contact  between 
the  groundeil  end  of  the  diode  and  a  pin 
on  one  of  tlu'  IC's.  It  was  deduced  that 
contraction  of  the  conformal  coating 
surrounding  the  diode  at  low  temperature 
had  resulted  in  cont.act  between  the 
mi'tal  body  of  the  diode  and  tlie  plated 
through  liole.  Corrective  action  w.as  to 
revise  the  CF.  Specification  to  call  for 
rai'tal  casi'd  parts  which  are  mounted  over 
printeil  <'ircuit.s  to  be  enclosed  in 
transparent  Insulation.  This  ]irablem 
h.as  not  recurred  thus  verifying  the 
I'f feet  iveness  of  the  corrective  action. 
F.iilure  analysis  participants  included 
.1  Qu.ilitv  Control  Engineer,  Design 
engineer,  I’ackaging  Engineer,  and  Mati— 
rials  and  Processes  personnel. 

Recentlv,  a  power  controller  failed 
due  to  excessive  output  from  the  voltage 
divider  on  a  printed  wire  board.  The 
problem  w.as  traced  to  a  resistor  which 
was  ri'turned  to  the  manufacturer  for 
further  analysis.  It  was  concluded  that 
there  was  contamination  consisting  of  a 
copper  fiber  internal  to  the  resistor 
introduced  by  a  copper  brush  used  to 
clean  the  cavity  of  the  production  mold. 
The  resistor  vendor  checked  all  product¬ 
ion  molds  and  there  was  no  evidence  of 
anv  copper  cleaning  tools.  Soft  hair 
brushes  only  may  be  used  for  cleaning 


per  standard  operating  procedures. 
Corrective  acti<'n  consisted  of  caution¬ 
ing  operators  against  using  anv  other 
cleaning,  tool.  Failure  analvsis  parti¬ 
cipants  included  a  Parts  Engineer, 
Quality  Control  Engineer,  Design  Eng¬ 
ineer,  the  Vendor's  Product  Engineer 
anti  Oualitv  Control  Engineer,  and  an 
outs  i  lie  1  .'ibora  torv  ,  utilixi'd  bv  the 
ventlor  to  identify  the  foreign  material 
inside  the  resistor. 

A  third  failure  analvsis  involving 
component  piece  parts  occurred  whiui 
Siliconix  FET  switches,  located  on  a 
Safe  Hold  Electronics  (SHE)  Control 
Amnlifier  PWB,  failed  to  open  .and  close. 
The  [>resenee  of  corroded  alnminii"i  wires 
and  bomling  p.ads  were  found  inside  the 
FET  switch.  X-rav  microprobe, 
x-ray  mapping,  and  residual  gas  analysis 
hy  GK  anil  Siliconix  laboratorv  consult¬ 
ants  verified  that  the  corrosion  was 
due  to  the  presence  of  chli'rine  inside 
the  FET  switch.  Intensive  analysis 
was  performed  to  determine  how  the 
chlorine  was  introduced  into  the  switch. 
It  was  concluded  that  the  chlorine 
entered  the  switch  during  board  cleaning, 
through  a  microcrack  in  the  eer.amic 
body  of  the  switch  introduced  during 
lead  forming  at  OF, .  The  problem  w.as 
traced  to  a  specific  operator  who  was 
reinstrncted  in  the  use  of  the  lead 
forming  tool.  All  FET  switches  inst.all- 
ed  by  this  operator  were  replaced  and 
inspectors  were  made  aware  of  the  prob¬ 
lem  and  are  monitoring  for  recurrence. 
Failure  analysis  participants  inclmleii 
a  Parts  Engineer,  Quality  Control 
Engineer,  Process  Control  Engineer, 
Manufacturing  Engineer,  the  Part.s  Test¬ 
ing  hah,  .Matori.als  and  Processes 
personnel,  the  Siliconix  Failuri'  Analv- 
si.s  Engineer,  two  outside  labs  utilised 
bv  CE  and  a  third  lab  utilixed  bv 
S  i  1  icon ix . 

tin  each  failure  analysis,  all 
participants'  efforts  arc  coordinated 
bv  a  Failure  Analvsis  Engineer  with  the 
entire  project  managed  by  the  Failure 
Analvsis  Board  Chairman. 
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liesides  lliu  mcLhods  discussed  above 
--  wide  distribution  of  failure  analvsis 
reports,  and  regular  Corrective  Action 
Hoard  and  Failure  Review  Board  nieetinp.s 
--  several  other  vehicles  provide  pos¬ 
itive  means  for  exchansinK  problem  and 
corrective  action  information.  Bv  far 
tlie  most  familiar  to  members  of  the 
Aerospace  community  is  the  GIDFP  Alert 
System.  The  major  problems  with  GIDFP 
tliough  are  the  potential  legal  ramifica¬ 
tions  which  do  provide  both  a  conscious 
and  subconscious  screen  into  tlie  quality/ 
<|uantity  of  GIDFP  Alerts. 

Other  customer  contacts,  both  writ¬ 
ten  and  verbal,  are  also  useful  in 
exchanging  information.  A  recent  example 
whereby  information  was  obtained  rapidly 
from  manv  different  sources  was  the 
Hevnolds  aluminum  problem.  Besides  the 
communications  received  from  various 
governmental  agencies,  Reynolds  personnel 
themselves  provided  much  information  and 
gu i dance . 

An  internal  method  vised  to  communi¬ 
cate  Quality  related  information  at 
General  Flcctric's  Space  Division  is  the 
Product  Ouality  Alert  (PQA) .  These 
Alerts  are  issued  periodically  based  on 
information  obtained  from  a  variety  of 
sources  including  GIDFP  Alert.s  and  the 
results  of  failure  analyses.  An  example 
of  a  Product  Quality  Alert  resulting 
f ri'm  a  failure  analysis  vyas  a  temperature 
sensor  (thermistor)  that  was  sliort  cir¬ 
cuited  due  to  an  internal  reflow  of  low 
melting  point  solder  which  shorted  both 
thermistor  contacts.  This  occurred 
during  soldering  at  GF.  of  the  short 
thermistor  leads  to  extensions.  The 
Product  (.lualitv  Alert  which  was  subse¬ 
quently  issued  defined  the  corrective 
actions  taken  on  the  specific  program 
involved  as  follows:  fire  anvl  post-pot 
resistance  checks;  mandatory  use  of  heat 
sinks  on  thermistor  leads;  [ilanning 
cautions  to  minimize  soldering  afipl  ica- 
tion  times;  and  a  post-fiot  isolation 
check  between  wire  leads  and  the  alumi¬ 
num  base  plate.  Of  extreme  importance 
was  the  closing  statement  of  the  Alert: 
"Al.l,  OTHFR  PROGRAMS  UTILIZING  THFRMISTOR- 
TYPF  TFMPFRy\Tl!RF  SFNSORS  SHOULD  ASSURF 
THAT  THF  ABOVF.  CORRECT  IVF  ACTIONS  ARE 


IMPLEMENTED  l.N  ORDER  TO  PRECLUDE  SIMILAR 
FAII.URF  MODES".  Naturally,  our  Product 
Oiialitv  Alerts  receive  widespread, 

.across  programs,  distribution. 

RECOMliENDED  IMl’ROVEMF.NTS 

Process  Readiness  is  a  management 
tool  that  has  been  used  bv  Gem'ral 
Electric's  Sp.ace  Division  for  ten  years. 
Its  purpose  is  to  produce  highly  reli¬ 
able  flight  hardware  repetitively  vyith- 
oiit  unusual  or  complex  quality  problems 
which  would  impact  cost  or  schedule. 

Just  as  an  increase  in  prevention  costs 
tends  to  lower  failure  costs,  expendi¬ 
tures  for  Process  Readiness  tend  to 
reduce  the  number  of  failure  analyst's 
required  on  a  program.  Process  Readi¬ 
ness  is  defined  as  a  management  action 
which  assures  that  process  development, 
documentation,  tooling,  equipment, 
facilities,  materials,  and  trained 
personnel  are  certified  "ready"  before 
use  of  the  process  for  the  production 
of  flight  hardware.  The  Process  Readi¬ 
ness  Team  consists  of  three  experienced 
functional  managers,  one  each  represent¬ 
ing  Quality,  Manufacturing,  and  Engi¬ 
neering.  The  same  team  members  servivv 
all  programs  to  assure  consistent 
implementation  of  process  readiness 
concepts  and  to  provide  an  independent 
unbiased  examination  of  program  require¬ 
ments.  During  the  initial  phases  of  a 
new  program,  the  Process  Readiness  Team 
interviews  senior  program  dedicated 
personnel  to  discuss  the  hardware 
required  and  the  proposed  processes 
necessary  to  produce  it.  Processes  that 
have  not  been  f^reviously  proven,  proven 
processes  that  have  a  significantly 
different  application  than  on  p.ist 
programs,  and  existing  processes 
currently  in  use  requiring  re-ex.amina- 
tion  are  candidatv's  for  a  thorough 
readiness  review.  A  Readiness  Plan  is 
then  prepared  for  each  identified 
critical  process.  The  Plan  is  divided 
into  the  following  five  s tanda rd i zod 
factors:  Process  Development;  Dc>cu- 

mentation;  Tools,  Equipment,  and 
Facilities;  Tr.iining;  and  Prime  Trvoul  . 
The  Readiness  Plan  utilized  in  this 
present.ation  w.as  prepared  for  a  s|iecial 
cable.  The  potting  m.aterial  h.id  been 
used  for  potting  on  past  programs  and 
personnel  had  experience  in  pin  weliling,. 
However,  due  to  the  cable  con f i g,ura t  i on 
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and  tlu'  application  of  tlieso  processes 
in  this  case,  certain  refinements  to 
the  pottlnK  and  welding  processes  were 
rei|uired.  Program  Process  Readiness 
mei'tings  aia-  usually  held  on  a  weoklv 
b.isis.  It  is  the  responsibility  of 
the  Process  Readiness  Team  to  assure 
the  completion  of  all  action  items  and 
to  demonstrate  this  to  the  satisfaction 
of  Proi'.ram,  ('ualitv  Manufacturing,  and 
liu- iiu‘er  inj;  Section  Managers.  This 
demons t ra t i on  often  includes  a  tour  of 
tile  facility  where  the  [Process  is 
performerl.  I'he  Section  Managers  then 
issue  a  ci'tti  f  ication  addressed  to  the 
ni'partmeiu  General  Manager  attesting 
that  the  process  will  produce  reliable 
flight  hardware  repetitively  without 
unusual  or  complex  problems  which  would 
lead  to  damaged  hardware,  high  cost, 
or  sliniied  schedules. 

ihe  Defect  Analysis  and  Trend 
Lvaluation  (D.A.T.E.)  System  is  a  new 
management  tool  being  utilized  by 
General  Electric.  Twenty-one  different 
eli-ments  from  each  nonconformance  report 
are  entered  into  a  computerized  system 
in  order  to  establisli  a  data  base  which 
in  turn  determines  defect  history. 

Nine  different  reporting  formats  have 
been  identified  and  of  these  nine, 
the  following  two  reports  are  most 
utilized:  sequenced  by  nonconformance 
ri'port  number  and  sequenced  by  drawing 
number.  More  than  3,500  NR's  have  been 
entered  into  the  data  base  within  the 
past  year  and  each  day,  more  and  more 
people,  including  customers,  are  making 
use  of  the  stored  data. 

ORGANIZATIONAL  RESPONSIBILITIES  FOR 
FA  lUfRE  A.NALYSIS 

Failure  analysis  should  be 
divorced  from  the  Program  Office. 
Responsibility  should  reside  in  the 
functional  organization  preferably 
within  either  Quality  Assurance  or 
Reliability.  It  is  felt  that  this 
would  result  in  the  following  benefits: 
a  more  independent  analysis;  a  dedicated 
team  to  service  all  programs  cost 
e f fee t ivc Lv ;  and  a  better  flow  of 
information  and  " 1 essons- 1  earned" 
across  programs. 


The  enstomer  should  not  be  directly 
involved  with  each  failure  analysis  but 
instead  should  receive  copies  of  all 
imbl ished  reports.  Failure  analysis 
results  sliould  be  reviewed  during 
customer/contractor  meetings,  e.g., 
Failure  Review  Board  (FRB)  and  Correc¬ 
tive  Action  Board  (CAB);  during  which 
time  customer  repriusentat i ves  are 
entitled  to  satisfactory  answers  to 
•any  questions  which  might  arise.  Bv 
far  the  most  important  role  for  the 
customer  is  to  serve  as  a  consultant 
since  some  customer  representatives 
have  extensive  experience  with  many 
contractors  .and  can  provide  constructive 
guidance  to  a  contractor  who  is  wrest¬ 
ling  with  a  "unique"  problem  which  in 
reality  has  already  been  solved  by 
another  contractor. 


FAILURE  ANALYSIS  IMPACT  ON  SYSTEM  RELIABILITY 


Ed  Doyle  3r. 

Program  Manager 

Quick  Reaction  Failure  Analysis  System  Support  Activity 
Rome  Air  Development  Center 
Griffiss  AFB  NY  13441 


The  effective  utilization  of  .the  Rome 
Air  Development  Center's  (RADC)^  capabili¬ 
ties  in  microelectronic  device  failure  analysis 
has  been  instrunental  in  improving  the  relia¬ 
bility  of  military  electronics.  The  Center, 
designated  the  Air  Force's  reliability  focal 
point,  maintains  a  Quick  Reaction  Failure 
Analysis  Laboratory  for  direct  reliability 
support  of  Air  Force  and  DoD  equipment  and 
systems  on  a  priority  basis.  Its  mission  is  the 
analysis  of  semiconductor  device  failures  in 
production  and  field  equipment,  the  identifica¬ 
tion  of  contributing  failure  mechanisms,  the 
recommendation  of  corrective  actions  and  the 
execution  of  system  fixes  on  a  quick  reaction 
basis.  The  technologiceil  base  for  the  RADC 
failure  analysis  expertise  is  derived  from  in- 
house  and  contractual  device  reliability  R&D 
programs,  experienced  technical  personnel, 
extensive  analytical  facilities,  an  existing 
corporate  memory  and  continuous  involvement 
in  diverse  system  support  projects. 

Microelectronic  device  failure  analysis, 
a  dynamic  and  vital  function,  provides  timely 
solutions  to  DoD  system  part  reliability 
problems.  In  resolving  these  device  reliability 
problems,  in-depth  failure  analysis  is  the  key 
task  leading  to  corrective  action  recommenda¬ 
tion,  implementation  and  evaluation.  Profi¬ 
ciency  in  this  area  not  only  requires  a 
thorougli  understanding  of  component  design, 
materials,  processing,  assembly,  packaging  and 
test  methods,  but  also  a  knowledge  of  device 
environmental  and  functional  failure  condi- 
tiais  --  important  factors  to  be  considered  in 
defining  a  failure  as  either  component  or 
application  related.  This  analysis  expertise, 
however,  comes  at  a  price.  Specialized 
analyst  training,  [periodic  facilities  updating 
and  experience  are  essential  to  make  available 
advanced  failure  analysis  techniques  and 
instrumentation  and  employ  these  tools  to 
meet  the  demands  of  today's  complex  semi¬ 
conductor  technology  and  assure  valid 
anal  yses. 

Because  the  overall  task  of  the  failure 
analyst  is  both  technically  and 


administratively  demanding,  it  is  important  to 
examine  this  key  study  and  its  management  in 
detail.  If  one  were  to  assess  the  technological 
accomplishments  in  the  field  of  microelec¬ 
tronics  over  the  past  fifteen  years,  one  might 
conclude  that  the  semiconductor  process 
specialist  usually  held  the  winning  hand,  and 
that  the  game  plan  of  the  failure  analyst  was 
basically  a  catch-up  strategy.  The  analyst  has 
been  forced  not  only  to  keep  abreast  of  a  fast 
moving  technology  with  its  proprietary  process 
innovations  such  as  device  manufacturing, 
testing,  and  applications,  but  also  to  investi¬ 
gate  new  developments  in  failure  analysis 
techniques  and  other  related  reliability 
subjects.  His  task  often  entails  solving  a 
device  problem  while  concurrently  deducing 
the  processes  employed,  a  reverse  engineering 
situation.  Contrasted  with  the  engineer  who 
designs  and  builds  new  equipment  ,  a  concrete 
goal,  the  analyst  must  be  highly  motivated  to 
pursue  a  somewhat  intangible  objective, 
improved  equipment  reliability.  To  make 
matters  worse,  this  expertise  is  usually  sought 
late  in  time  during  equipment  production  or 
after  deployment,  when  other  attempts  to 
correct  problems  have  failed.  This  coming-on¬ 
board  downstream  is  due  in  part  to  the 
time/ temperature  dependency  of  certain 
failure  mechanisms,  among  other  reasons. 
Obviously,  this  minimum  time  to  react, 
without  causing  unnecessary  production  delays 
or  operational  down-time,  adds  to  the  pressure 
of  effecting  a  fix. 

The  critical  step  for  attaining  improved 
device  reliability  is  corrective  action  feedback 
to  the  part  vendor.  Here,  the  data  gathered 
by  device  test  and  analysis,  and  other  prior 
experience,  is  applied  in  devising  a  viable 
solution  for  rectifying  the  specific  part 
problem.  Corrective  action,  based  on  part 
test  and  analysis  findings,  may  take  the  form 
of  a  design,  material,  process  or  test  change 
and,  occasionally,  circuit  application  modifi¬ 
cations.  Design  and  fabrication  changes 
usually  piertain  to  the  device  itself.  The 
potential  reliability  improvement  must  then  be 
verified  by  further  test  and  evaluation.  A  test 


change,  assessed  on  a  completed  product,  may 
consist  of  specifying  either  additional 
parametric/functional  tests  or  new  screen 
tests  tailored  to  a  specific  failure  mechanism. 
It  may  also  be  necessary  to  recommend  a 
device  change  when  process  incompatibilities 
exist  or  when  effective  screens  cannot  be 
devised,  are  impractical  or  are  too  expensive. 

Finally,  the  corrective  action  implemen¬ 
ted  is  often  governed  by  system  constraints 
such  as  cost,  delivery,  performance,  configur¬ 
ation  and  ease  of  retrofit.  Part  availability 
may  also  be  a  significant  factor.  Each  reli¬ 
ability  fix  must  be  assessed  on  an  individual 
basis,  taking  into  account  all  system  require¬ 
ments  and  corrective  action  alternatives. 

System  support  priorities  and  device 
criteria  must  be  imposed  to  effectively 
manage  a  failure  analysis  activity  within  an 
R&n  organization.  The  criteria  used  by 
i'iAOC  for  decisions  on  committing  manpower 
and  resources  for  system  support  is  briefly 
outlined  below. 

Criter ia  for  Failure  Analysis 
System  Support 

Customer  Priorities 

0  Air  Force 

o  AFSC  System  Product  Divi¬ 

sions  &  Contractors 
o  AFSC  Laboratories  &  Test 

Centers 
o  AFLC 

o  Department  of  Defense 

o  Otfier  Government  Agencies 

o  Industry 

Support  Criticality 

o  System  Priority 

o  Device  Technology  Involved  (Semi¬ 

conductor  Technology) 

o  Device  Usage 

o  Critical  System  Components 

o  Reliability  Data  Requirements 

(Failure  Modes/Mechanisms) 


Selected  criteria  warrant  further  explan¬ 
ation.  As  indicated,  semiconductor  technology 
is  emphasized.  Since  field  failure  analysis 
efforts  usually  involve  yesterday's  device 
technology,  in  varying  degrees,  and  system 
designs,  due  consideration  is  given  to  the  need 
versus  benefit  tradeoff.  Flere,  current  or 
anticipated  high  volune  device  usage  is  a 
factor  assessed.  Lastly,  critical  system 
components  often  involve  one  of  a  kind 
devices,  small  volume  usage,  and  occasionally, 
non-semiconductor  components,  all  vtilid 
points  to  appraise  prior  to  system  support 
commitment. 

After  accepting  a  system  support  job, 
there  are  several  factors  the  failure  analysis 
activity  manager  considers  in  planning  and 
executing  the  effort.  These  factors  are 
presented  briefly  below. 

o  Determine  resources  required  such 

as  manpower,  travel  funds,  and 
supplies  and  equipments. 

o  Assign  appropriate  personnel  to  the 

technical  analysis  area  involved. 

o  Establish  definite  schedules  for 

analysis  completion  and  the 
preparation  of  a  technical  report 
based  on  existing  system  scliedules. 

o  Estimate  additional  time  required 

for  any  follow-up  actions  and 
briefings  in  light  of  other  c'urrent 
or  anticipated  system  support 
commitments. 

o  Adjust  job  schedules  periodically 

based  on  existing  workload  and 
system  priorities. 

o  Plan  w'orkload  to  allow  some  flex¬ 

ibility  for  handling  unanticipated 
high  priority  system  support 
requests. 

o  Interfacing  with  people  at  all 

levels  of  authority  and  maintaining 
a  fXDsitive,  aggressive  attitude 
toward  assuring  the  reliability  of 
military  defense  systeins  at 
reasonable  cost,  without  sacrific¬ 
ing  performance. 

Where  the  failure  analysis  activity  is  an 
integral  part  of  the  system  support 
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organizaticxi,  as  is  the  case  at  RADC,  the 
analysis  team  is  faced  with  demanding  follow¬ 
up  actions.  The  job  doesn't  end  after 
determining  the  failure  mode/mechanism  and 
suggesting  a  fix.  Effective  corrective  action 
implementation  involves  many  other  factors: 

o  Convincing  system  managers  that 
the  recommended  action  is  necessary.  This 
implies  being  prepared  to  defend  analysis  find¬ 
ings  and  recommendations  at  all  levels  of 
authority.  Sound  technical  efforts  can  be 
negated  by  poor  salesmanship. 

o  Relaxing  defensive  attitudes  on  the 

part  of  any  one  or  all  parties  concerned.  The 
"not  invented  here"  hangup,  or  "it's  the  other 
guys  fault"  disposition,  must  be  diplomatically 
addressed  when  third  parties  are  engaged  for 
performing  analysis  and  recommending  solu¬ 
tions.  A  positive  attitude  reflecting  a  cooper¬ 
ative  effort  in  problem  solving  can  ease 
tensions. 

o  Overcoming  inertia,  resistance  to 

change,  when  implementing  corrective  action 
that  interrupts  production  flow,  delays 
delivery  or  increases  costs.  Persistence  is 
essential  throughout  the  corrective  action 
transition  period. 

o  Acting  as  a  mediator  between  the 
responsible  government  agency  and  the  vendor 
or  contractor.  Each  party  has  a  vested 
interest  in  the  outcome  of  any  action  taken. 
Industry  wants  to  make  a  good  product,  but 
must  make  a  profit  to  stay  in  business. 
Government  agencies  have  the  responsibility 
to  spend  the  taxpayer's  money  wisely  without 
sacrificing  system  reliability  or  performance. 

o  Recognizing  the  sometimes  over¬ 

riding  factors.  Management  conflicts  arise 
because  of  unrealistic  or  absent  equipment 
reliability  specifications,  existing 

performance/reliability  tradeoffs,  limited 
funding  and  proprietary  data  claims,  to  name  a 
few. 

o  Overcoming  contractor  reluctance 

to  admit  questionable  reliability  practices  like 
using  non-standard/non-military  specified 
devices  to  reduce  cost,  sole  source  "critical 
device"  procurements,  inter  divisional  subcon¬ 
tracting,  "keep  it  in  the  family"  tendencies, 
and  designs  using  new  devices  with  unproven 
reliability. 


These  cited  pursuits  are  unavoidable  for 
assuring  that  recommended  corrective  actions 
are  brought  to  the  attention  of  decision  mak¬ 
ing  authority,  technically  understood,  and 
properly  weighed  against  other  system 

constraints.  The  impact  of  corrective  action 
on  system  reliability  is  realized  only  when 
implementation  has  been  accomplished  and 
ev^uated.  The  experienced  and  effective 

failure  analysis  activity  manager  must  be 
aware  of  these  concepts  and  continually  strive 
to  maintain  the  degree  of  system  visibility 
needed  to  accomplish  this  formidable  job. 

Recognizing  that  the  management  of  an 
effective  failure  analysis  system  support 

activity  presents  a  real  administrative 

challenge,  let  us  now  turn  to  the  failure 
analysis  laboratory  environment  and  address 
the  technical  dilemma  which  semiconductor 
device  technology  presents  to  managers  and 
analysts  alike.  Five  main  steps  are  blocked 
out  from  the  definition  of  a  device  reliability 
problem  to  a  completed  system  fix: 

o  Failure  Verification 

o  In-Depth  Failure  Analysis 

o  Corrective  Action  Definition 

o  Corrective  Action  Implementation 

o  Corrective  Action  Evaluation 

Failure  verification  involves  functional 
and  environmental  testing  to  determine 
whether  a  device  is  an  actual  failure,  inter¬ 
mittent  failure,  or  retest  good.  Field  repair 
technicians  often  replace  several  components 
before  correcting  equipment  malfunction 
because  fault  diagnostic  procedures  often 
don't  isolate  failure  at  the  part  level.  The 
device  may  also  be  temperature  sensitive, 
recovering  after  power  or  temperature  condi¬ 
tions  are  removed  ,  which  requires  further 
verification  testing.  Operating  the  device 
with  power  applied  at  temperature  for  extend¬ 
ed  periods  may  be  necessary.  Thus,  classifying 
a  suspected  part  failure  as  "retest  good"  often 
means  more  test  time  than  verifying  an  actual 
failure. 

During  the  failure  verification  phase,  a 
review  of  device  operational  and  failure 
history  can  aid  in  selecting  the  most  appropri¬ 
ate  tests  to  perform.  This  is  accomplished  by 
direct  contact  with  cognizant  personnel, 


whenever  possible,  and  by  the  use  of  ttie 
illustrated  RA13C  System  Support  Request 
Form  provided  to  authorized  OoD  requesters. 


to  [xovide  useful  data.  Whether  device  failure 
conditions  are  provided  or  not,  the  analyst 
must  always  make  a  determination  of  whether 
the  failure  is  a  device-related  problem,  system 
design  deficiency,  or  environmentally  induced. 

In-depth  failure  analysis  attempts  to 
identify  the  device  failure  mode  and  failure 
mechanism.  Electrical  failure  modes  are 
usually  classified  as  opens,  shorts,  or  param¬ 
eter  out  of  specification.  The  failure  mode 
degree  can  be  categorized  as  catastrophic, 
degradation,  or  intermittent.  The  failure 
mechanism  is  the  basic  chemical  reaction  or 
physical  change  associated  with  the  device, 
which  results  in  a  failure  mode  recognizable 
by  measurement.  Similar  definitions  can  be 
assigned  to  mechanical  failure  modes  and 
mechanisms. 

The  illustrations  shown  convey  at  a 
glance  the  magnitude  of  the  analyst's  problem 
in  dealing  witfi  rapid  technology  advances. 
Both  the  design  engineer  and  system  analyst 
have  been  forced  to  use  digital  rather  than 
analog  techniques  for  implementing  most 


system  functions.  The  proliferation  of  device 
functional  types,  particularly  microcircuits, 
was  attractive  to  designers  from  an  increased 
performance  standpoint.  Understandably,  new 
devices  were  designed  into  equipment  before 
their  reliability  was  fully  assessed.  The  spec 
writer  and  the  analyst  tried  valiantly  to  keep 
pace. 


These  diverse  microcircuit  electrical 
functions  evolved  from  novel  design  concepts 
and  the  development  of  new  processes.  The 
variety  of  microcircuit  generic  types  has 
constantly  increased,  each  having  certain 
advantages  over  other  types.  Examples  are 
ECL(speed),  C\10S(low  power  consumption), 
MOSdiigh  input  impedance),  low  power 
Schottkydmproved  speed-power  product  when 
compared  to  standard  TTL  logic).  Factors  like 
the  non-volatility  of  VINOS  memories  when 
compiared  with  TTL  and  MOS  product,  the 
improved  radiation  hardness  of  devices  using 
dielectric  isolation,  and  the  advances  in 
microwave  solid  state  devices  have  forced  the 
analyst  to  expand  his  focus  on  technology 
frequently.  It  should  be  apparent  that  the 
failure  analyst  must  have  a  working  knowledge 


of  many  device  types  and  their  processing 
differences  to  maintain  a  quick  reaction 
response  in  the  system  support  environment. 
Staying  on  top  requires  a  balanced  R&D 
program  in  device  technology  and  reliability. 


LSI  devices.  In  the  case  of  microprocessors 
(lower  photos),  where  isolation  of  failure  at 
the  element  level  may  not  be  feasible,  other 
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Concurrently,  an  even  more  demanding 
task  has  been  the  development  of  new  and  the 
refinement  of  established  failure  analysis 
techniques  to  meet  the  challenge  of  increasing 
microcircuit  functional  element  density.  The 
advent  of  the  three  and  four  inch  wafer 
processing  technology  coupled  with  the  reduc¬ 
tion  of  device  geometries  on  the  chip,  made 
possible  by  improved  process  control,  has 
resulted  in  the  current  LSI  memories,  micro¬ 
processors  and  complex  peripheral  devices. 
These  highly  complex  devices  pose  a  major 
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problem  in  failure  verification,  and  in-dcpth 
analysis  has  quickly  become  a  monumental 
task.  Sophisticated  electrical  faLilt  isolation 
techniques  using  computer  test  programs, 
compatible  with  existing  automated  test 
equipment  (upper  right  photo),  are  needed 
today  to  isolate  a  degraded  circLiit  element  in 


supplemental  techniqu^  ^  ^uch  as  nematic 
liquid  crystal  analysis,^’  ’  which  optically 
highlights  operating  circuit  paths  in  a  manner 
similar  to  digital  watch  displays,  are  being 
refined  to  provide  additional  fault  isolation 
capabilities. 


i  i 


In  the  failure  analysis  area,  as  device 
complexity  increased  negating  the  use  of  some 
established  techniques,  selected  analysis 
methods  were  further  refined  and  new 
techniques  were  developed.  Analytical  tools 
such  as  photovoltage  response  measurements, 
stereo  x-ray  and  neutron  radiography, 
Nomarski  interference  contrast  microscopy 
for  surface  topography  enhancement,  electron 
and  ion  beam  instruments,  mass  spectrometer 
residual  gas  analyzers,  infra-red  thermal 
profilers,  and  other  powerful  analysis  methods 
surfaced.  The  scanning  electron  microscope 
(upper  photo)  provides  high  magnification 
displays  with  exceptional  depth  of  field  of 
submicron  structures  beyond  the  resolution  of 
optical  tnicroscopy.  Another  electron  beam 
instrunent,  an  electron  probe  microanalyzer 
(lower  photo)  performs  x-ray  spectrochemical 
analysis  for  identifying  the  elemental 
constituents  of  solids  in  a  cubic  micron 
vol  une. 

It  was  also  readily  apparent  that  these 
powerful  analysis  instruments  required 
extensive  [personnel  training  and  experience. 


and,  in  some  cases,  were  limited  in  application 
and  subject  to  data  interpretation.  These  two 
electron  beam  analytical  instruinents,  along 
with  Auger  spectroscopy  for  chemically 
analyzing  material  layers  by  depth  profiling, 
are  widely  used  for  failure  analvsis  of  today's 
microelectronic  devices.  Thus,  to  successfully 
identify  device  failure  mechanisms  and  resolve 
part  and  equipment  reliability  problems,  the 
combined  data  obtained  by  employing  several 
techniques  is  often  necessary.  The  test  and 
analysis  facilities  required  for  failure  analysis 
of  today's  microelectronic  devices  has  also 
demanded  increasing  capital  investments  for 
functional  test  and  analytical  instrumentation. 
The  simple  but  effective  failure  an^'  s 
diagnostic  tools  of  yesterday  must  oe 
constantly  updated  to  meet  the  demands  of 
complex  LSI  technology. 


In  an  attempt  to  provide  the  failure 
analyst  with  an  updated  compendium  of  avail¬ 
able  failure  analysis  techniques,  RADC  is 
preparing  a  "Microelectronic  Device  Failure 
Analysis  Procedural  Guide".  The  contents  of 
the  "Guide"  will  cover  both  routine  analysis 
metliods  and  advanced  techniques  as  presented 
m  the  following  outline. 

I.  General  Introduction 

II.  Reference  Documents  Available 

III.  Failure  Analysis  Techniques 

IV.  Laboratory  Safety  Procedures 

V.  Glossary  of  Terms 

VI.  Glossaryof  Materials 
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Vll.  Failure  Analysis  Technique 
References 


The  "Guide"  will  describe  the  application  of 
analysis  techniques  for  the  detection,  identifi¬ 
cation  and  measurement  of  microelectronic 
device  failure  modes  and  mechanisms,  using 
typical  failure  analyses  as  examples. 

The  technical  talent  necessary  for 
failure  mechanism  studies  includes  personnel 
with  backgrounds  in  electronics,  physics, 
optics,  chemistry  and  metallurgy.  Having 
defined  "failure  mechanism"  as  a  chemical 
reaction  or  physical  change  resulting  in  device 
failure,  the  analyst  must  then  determine  the 
many  interdependent  factors  which  can 
influence  the  observed  reaction  and  the  rate 
at  which  it  proceeds.  The  analyst  must  also 
recognize  that  any  one  or  a  combination  of 
factors  such  as  wafer,  processing,  and  packag¬ 
ing  defects,  time  dependent  reactions  or 
changes,  and  environmental,  externally 
induced,  test  or  hrman  error,  can  result  in 
device  functional  failure.  Each  technology 
also  has  its  unique  problems. 

The  following  typical  case  history 
involved  corrosion  of  internal  aluminum  lead 
wires  in  hermetic  package  microcircuits  used 
in  an  avionics  computer.  A  typical  corrosion 
failure  is  depicted  in  the  following  photo. 

Computer  malfunction  was  traced  to 
these  microcircuits  which  exhibited  open 
internal  lead  wires.  The  failures  occu'red 
three  years  after  computer  installation  in 
operational  aircraft  and  continued  to  exhibit 
an  excessive  failure  rate  thereafter.  Back¬ 
ground  information  gathered  indicated  that  a 


non-standard  dye  penetrant  hermeticity  test  was 
performed.  This  additional  leak  test  was  felt 
necessary  because  of  previously  documented 
hermeticity  test  escapes.  Failure  analysis 
revealed  that  excessive  moisture  was  trapped 
in  the  package  cavity  due  to  inadequate  pre¬ 
firing  of  the  lid  seal  glass  and  that  the  dye 
penetrant  fluid  used  contained  significant 
amounts  of  chlorine  impurity.  Further 
analysis  verified  that  long  term  anodic  corro¬ 
sion  of  the  aluminum  wires,  as  evidenced  by 
the  presence  of  a  hydrated  form  of  aluminum 
oxide  at  the  wire  break  site,  was  caused  by  the 
chemical  reaction  of  chlorine  ions  with  the 
aluminum  wire  in  the  presence  of  excess  water 
vapor.  The  reaction  rate  was  found  to  depend 
on  package  moisture  levels,  chlorine  concen¬ 
trations,  ambient  temperatures  and  environ¬ 
mental  vibration  conditions. 


Although  the  actual  part  failure  analysis 
was  straightforward,  the  total  problem 
magnitude  and  solution  was  obtained  by 
meticulous  attention  to  details.  This  meant 
getting  an  accurate  part  failure  history  from 
the  system  p>rime  and  equipment  contractors, 
reviewing  the  part  vendors  processing  and 
screening  data  and  even  inspecting  sus[x-(  t 
microcircuits  from  failed  computers  at  th- 
flight  line  repair  facility  as  they  o<  curred. 
Finally,  a  substantial  corrective  action 
program  was  pursued  until  a  sucri'sst  il 
computer  fix  was  made. 


Military  munition  fuze  manufacturers 
often  elected  to  use  plastic  transistors  and 
microcircuits  because  of  their  superior 
mechanical  properties  in  severe  high-G  drop 
stock  and  vibration  environments.  The  cost 
advantage  of  these  devices  has  also  been 
quoted  as  an  economic  factor  supporting  their 
use  for  this  application.  Thus,  the  joint 
AF/contr  actor  decision  to  procure  large 
quantities  of  plastic  devices  for  most  fuze 
designs  during  the  Southeast  Asia  conflict  was 
considered  cost  effective  with  improved 
performance  in  the  operational  environment. 
This  action  appeared  justifiable  when  fuze 
production  was  geared  to  immediate  use  with 
ordnance,  but  stockpiling  these  fuzes  for 
indefinite  periods  after  the  SEA  conflict 
created  other  problems.  The  known  long  term 
reliability  problems  associated  with  injection 
molded  plastic  encapsulated  devices,  ncimely, 
moisture  penetration  under  high  humidity 
storage  and  fractured/lifted  internal  wire 
bonds  under  tempierature  cycling  conditions 
were  underestimated.  Chip  metallization 
corrosion  and  open  lead  wire  failures  were  the 
penalty.  The  absence  of  a  metallurgical  bond 
at  the  plastic  material/lead  frame  interface 
and  the  thermal  mismatch  (differential 
expansion)  between  the  internal  fly  wires  and 
surrounding  plastic  material  are  the  root  of 
the  reliability  problems  associated  with  these 
devices.  Temperature  cycling  generated  lead 
wire  failures  as  illustrated  in  the  photo  below 
arc  also  accelerated  by  out  of  control  bonding 
operations  (high  bonding  pressure)  resulting  in 
excessive  bond  squashout  and  lifting  (upper 
right  photo).  These  cited  reliability  problems, 
though  reduced  by  improved  materials  and 
processes,  still  exist  in  today's  plastic  devices. 
Stored  munition  fuzes,  periodically  sampled 
for  functionality,  have  exhibited  high  failure 
rates  traced  by  in-depth  analysis  to  the  failure 


mechanisms  described.  Therefore,  based  on 
existing  field  data,  the  current  10-20  year 
fuze  storage  requirement  demands  the  use  of 
hermetic  devices  specified  for  military  appli¬ 
cations.  RADC's  continued  reliability  assess¬ 
ment  of  plastic  device  technology  and 
sustained  support  efforts  to  assure  the  use  of 
mil-spec  devices  in  current  munition  and  fuze 
production,  wherever  possible,  are  directed 
toward  preventing  a  reoccurrence  of  past 
reliability  problems.  For  example,  the  Laser 
Guided  ^mb  (LGB)  initial  production  units 
employed  plastic  devices  with  resulting  poor 
demonstrated  reliability.  Current  production 
units  LBe  a  high  percentage  of  hermetic  mil- 
spec  devices,  a  positive  action  toward  upgrad¬ 
ing  the  reliability  of  an  expensive  but  effec¬ 
tive  tactical  weapon. 

Equipment  device  reliability  problems 
are  more  tolerable  in  ground  and  airborne 
environment  applications  since  failures  can 
eventually  be  repaired  even  though  these 
systems  may  be  operated  for  some  intermed¬ 
iate  period  in  a  reduced  capacity  mode.  In 
space  and  missile  system  applications, 
however,  on-board  device  failures  present  a 
more  critical  problem.  Depending  on  the 
degree  of  built-in  redundancy  with  its  inherent 
size-weight  vs.  reliability  tradeoff,  device 
failures  can  result  in  total  mission  failure  at  a 
substantial  cost.  Thus,  "design  for  reliability" 
principles  and  the  use  of  established  reliability 
devices  are  essential  system  procurement 
practices. 

In  addition  to  those  device  failure  modes 
common  to  any  application,  another  failure 
cause  encountered  in  space/missile  applica¬ 
tions  involves  unattached  particles  within 
device  packages  that  are  free  to  move 


randomly  in  zero-G  environments  and  under 
the  influence  of  electrostatic  forces  within 
the  device  package.  Depending  on  their 
physical  size,  these  particles  can  damage 
internal  lead  wires  and  chip  passivation  upxDn 
contact  and,  if  conductive,  can  result  in 
electrical  shorts  when  lodged  between  lead 
wires  and  non- passivated  chip  metallization  as 
illustrated  in  the  following  photos.  In  this 


equipiment  fix  prior  to  future  scheduled 
launches. 

Another  similar  event  involved  conduc¬ 
tive  particles  in  a  pxDwer  transistor  package 
due  to  metal  cover  weld  splash  in  the  absence 
of  a  protective  internal  barrier.  Depicted 
below  are  packages  with  and  without  this 
internal  barrier  for  comparison  (upper  photo) 
and  a  typical  weld  splash  (lower  pihoto).  The 
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case,  several  rnicroarcuit  failures  due  to 
minute  lid  seal  solder  particles  were  detected 
during  prelaunch  system  checkout  testing. 
After  assessing  the  reliability  risk  and  all 
possible  alternatives  for  corrective  action 
commensurate  with  existing  satellite  launch 
schedules,  a  decision  was  made  to  launch 
without  activating  payload  electronics  until 
orbit  was  achieved.  This  procedure  minimized 
possible  catastrop)hic  satellite  electronics 
failure  caused  by  freeing  loosely  attached 
conductive  particles  under  launch  vibration 
condtions,  which,  with  power  applied,  could 
result  in  hard  circuit  shorts  and  lead  to  further 
secondary  device  failures.  Follow-up  device 
corrective  actions  were  implemented  to 
increase  the  fly  wire  angle  with  respect  to  the 
chip  surface  and  provide  adequate  f>assivation 
over  the  chip  large  area  pjeripheral  metalliza¬ 
tion.  Here,  a  single  source  for  the  micro- 
circuit  added  to  the  time  for  effecting  a  total 


contractor  overlooked  published  Government- 
Industry  Data  Exchange  Program  (GIDEP) 
Alerts  identifying  this  problem  associated  with 
several  transistor  vendors  and  subsequently 
failed  to  remove  devices  in  eq'jipxnent  prior  to 
launch.  Failure  of  the  guidance  system  caused 
the  loss  of  a  $50\1  Titan  III  launch  vehicle 
with  payload.  After  isolating  guidance  system 
failure  to  these  defective  transistors,  the 
contractor  issued  a  purge  stock  order  and  an 
equipxnent  retrofit  corrective  action  for 
transistor  substitution. 

The  source  of  foreign  material  ran  be 
package  constituent  particles,  (o.g.,  the  nickel 
flake  shown  in  the  following  photo  whn  h 
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s'lortrJ  a  transistor  under  vibration 
conditions),  pieces  of  cleaved  silicon  from  the 
c|up  Itself,  wire  fragments,  portions  of 
eutectic  chip  attach  (balling  and  flaking)  and 
other  external  origin  material  introduced 
during  processing  and/or  packaging.  Their 
presence  is  attributed  to  the  lack  of  process 
control  and  inadequate  quality  assurance 
screening  inspections.  Today,  additional  M  00‘i'o 
reliability  screen  tests,  including  x-ray  and 
[lartick'  impact  noise  detection,  are  specified 
lor  "Class  S"  devices  used  in  space 
applications. 

Other  examples  of  device  failures 
experienced  in  military  equipment  applications 
include  (1)  intermittent  diodes  (cracked 
pellets)  due  to  stress  caused  by  an  off- 
centered  pellet  or  axial  lead  misalignment 
during  assembly  (F’hoto  a),  (2)  electrical  opens 
in  microcircuit  aluminum  metallization  films 
caiiseil  by  electromigration  effects  at  high 
f  iirrent  densities  ([’hoto  b),  (3)  transistor 

internal  wire  bond  lift  failures  due  to  minimal 
reacted  bond  area  (Photo  c),  and  (4)  hybrid 
device  wire  bond  degradation  and  correspond¬ 
ing  low  bond  strength  caused  by  diffusion 
couple  voids  accompanying  the  Kirkendall 
effect  (Photo  d).  These  devices  were  incor¬ 
porated  in  an  avionics  flight  control  computer, 
ground  radar  signal  processor,  missile  guidance 
system  and  space  reentry  vehicle  electronics 
respectively,  and  required  extensive  analyses 
and  corrective  actions  in  providing  quick 
reaction  equipment  fixes. 
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These  case  studies  involved  simple  tran¬ 
sistor  and  TTL  microcircuit  structures  where 
sophisticated  fault  isolation  diagnostics  were 
not  required.  Analysis  of  today's  complex 
devices,  however,  requires  developing  comput¬ 
erized  electrical  test  vector  fault  routines  for 
failure  verification  and  isolation.  Thus,  the 
test  and  analysis  time  is  rapidly  becoming  a 
larger  percentage  of  the  total  manhours 
expended  in  providing  quick  reaction  equip¬ 
ment  fixes.  If  we  fail  to  provide  timely 
analysis  on  today's  cotnplex  circuits,  the  feed¬ 
back  loop  to  the  manufacturer  will  be  tempor¬ 
arily  opened  and  necessary  corrective  actions 
delayed.  The  system  user's  alternative,  to 
maintain  a  readiness  posture,  may  be  a  direct 
replacement,  on  an  as-failed  basis,  of  parts 
from  the  same  vendor  or  another  vendor's 
"equivalent"  part.  Extensive  spares  inventorv 
and  repair  actions  are  expensive  when  field 
failures  are  involved.  Obviously,  the  system 
designer's  alternative  is  to  use  reliability 
proven  device  technologies. 

The  failure  modes  and  mechanisms  in 
current  technology  devices  are  not  necessarily 
new,  just  harder  to  detect,  isolate  and 
analyze.  Some  new  problem  areas  have 
surfaced  with  multi-level  metal/dielectric 
structures,  in  chip  protective  networks,  and 
soft  failures  including  alpha  particle  effects  in 
memory  devices  among  others,  which  can  only 
be  solved  by  aggressive  R3cD  programs. 

Updating  one's  knowledge  of  factors 
contributing  to  part  failure  requires 
substantial  homework,  experience,  and  a  good 
corporate  memory.  The  latter  involves 
adequate  documentation  of  analysis  findings 
when  working  in  the  system  support  environ¬ 
ment.  RADC  uses  both  a  short  form  as  shown 
in  the  "RADC  Failure  Analysis  Report,"  in 
addition  to  detailed  reports  which  are 
published  in  periodic  "RADC  In-Depth  Failure 
Analysis  Quick  Reaction  (QR)  Reliability 
System  Support  Accomplishments"  rf'oorts^ 
available  to  DoD  agencies. 

In  planning  for  the  future,  RADC  contin¬ 
uously  updates  its  facilities  to  meet  the  antic¬ 
ipated  analytical  demands  of  tomorrow's 
microelectronic  technology.  The  Center  also 
maintains  a  Cooperative  Education  Program 
with  several  engineering  universities,  provid 
ing  RADC  a  potential  source  of  personnel  with 
the  latest  engineering  technology.  Those 
assigned  in  the  device  reliability  area  are 


trained  on  the  job  in  the  disciplines  of  reliabil¬ 
ity,  semiconductor  technology,  testing,  failure 
analysis  and  system  support  activities. 

The  impact  of  RADC's  failure  analysis 
system  support  activity  has  been  realized  by 
the  improved  reliability  of  Air  Force  systems. 
These  accomplishments  range  from  short  term 
equipment  fixes  to  long  term  life  cycle  cost 
benefits.  The  time  element  for  effecting  a 
system  fix  can  range  from  a  few  days,  weeks 
or  months,  to  more  extended  tasks  spanning 
several  years.  The  ASD  RIVET  GYRO  Program 
which  sought  the  improvement  of  the  field 
reliability  of  operational  avionics  equipment, 
and  the  current  Missile-X  (MX)  Program  arc 
typical  examples  of  long  term  RADC  reliabil¬ 
ity  support. 

Through  a  sustained  cooperative  effort 
with  the  equipment  contractor,  RADC  accom¬ 
plished  a  two  order-of-magnitude  increase  in 
the  MTBF  of  the  F-4  Integrated  Display 
System  (IDS)  by  design  and  component  correc¬ 
tive  action  implementation.  Recent  RADC 
efforts  in  support  of  the  SRAM  Master 
Computer  have  resulted  in  a  net  cost  avoid¬ 
ance  of  $10M  through  in-depth  microcircuit 
failure  mechanism  identification  and 
replacement  spares  acquisition  actions.  Other 
ASD  avionics  equipments.  ESD  ground 
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systems,  and  SAMSO  (now  two  separate 
organizations  designated  SD  and  BMO)  missile 
and  satellite  programs  have  been  successfully 
supported,  as  have  many  ADTC  munition 
programs.  The  system  support  activities 
described  also  provide  timely  guidance  in 
formulating  the  Center's  reliability  R&D 
program  and  pertinent  data  for  updating 
military  device  reliability  specifications  and 
standards. 

Contractually,  RADC  investigated 
microcircuit  failures  from  production  and  field 
F-15  Head-Up  Display  (HUD)  and  Interfer¬ 
ence  Blanker  Sets  (IBS)'’.  This  effort  was 
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undertaken  to  access  the  reliability  of  current 
microcircuit  technology  in  the  avionics 
environment  and  the  validity  and  effectiveness 
of  presently  used  techniques  for  microcircuit 
procurement,  screening  quality  assurance,  and 
reliability  prediction.  The  findings  verified  a 
3:1  demonstrated  failure  rate  improvement  of 
M1L-\1-3S510  microcircuits  over  vendor  MIL- 
STD-8S3  equivalent  devices,  thus  validating 
the  effectiveness  of  current  microcircuit 
procurement  documents. 

The  high  percentage  of  system  device 
(leld  failures  duo  to  electrical  overstress 
prompted  a  recently  completed  I'lAl'X^ 
contractual  effort  with  BDM  Corporation 
din;(  ted  toward  developing  a  microcircuit 
"Electrical  Overstress  Tolerance  Qualification 
Test".  The  "Electrostatic  Discharge  Sensitiv¬ 
ity  Test,  Method  30XX,"  is  currently  in 
government/industry  coordination  for  inclusion 
in  Method  3005  of  MIL-ST13-883B  and  MIL-.M- 
38  5  10. 

The  overall  impact  of  RADC's  system 
support  activities  through  in-depth  device 
tailiire  analysis  has  been  far-reaching.  Besides 


providing  timely  system  fixes  and  guidance  to 
the  Center's  Reliability  Technology  RdcD 
program,  the  device  problems  surfaced, 
analyzed  and  resolved  have  resulted  in  signifi¬ 
cant  inputs  to  military  device  specifications 
and  standards,  namely,  .MlL-M-385 10,  "General 
Specifications  for  Microcircuits”  and  MIL- 
STD-883,  "Test  Methods  and  Procedures  for 
Microelectronics". 

The  current  contractual  and  in-house 
program  for  compiling,  editing  and  publishing 
an  "RADC  Microelectronic  Failure  Analysis 
Procedural  Guide"  will  provide  government 
and  industrial  organizations  with  up-to-date, 
comprehensive  techniques  useful  for  personnel 
training  while  improving  the  experienced 
analyst's  knowledge  and  means  of  implement¬ 
ing  new  techniques  such  as  Scanning  Acoustic 
Microscopy  and  Nematic  Liquid  Crystal 
Displays  for  fault  isolation  diagnostics  in 
complex  LSI  technology. 

The  improvement  of  microelectronic 
device  reliability  and  the  attainment  of  reli¬ 
able  militarv  electronic  systems  depends  on 
the  continued  visibility  afforded  by  sustained 
involvement  in  system  support  microelectronic 
device  failure  analysis  activities.  It  is  RADC's 
goal  to  stay  at  the  forefront  ol  solid  state 
technology  and  its  system  applications  m  this 
important  reliability  area. 
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FAILURE  ANALYSIS  IN 
THE  DESIGN  PHASE  OF  A 
HI-REL  AEROSPACE  SYSTEM  - 
AS  VIEWED  BY  A  CUSTOMER 
CONSULTANT 

J.  R.  Howell 
Systems  Effective 
Project  Engineer, 

The  Aerospace  Corporation 


SUMMARY; 

Failure  analysis  is  a  most 
important  concept  for  mission 
assurance.  In-depth  failure 
analysis  identifying  the  mode  of 
failure  and  the  exact  mechanism 
which  produced  the  failure  (i.e, 
physics  of  failure)  provide 
program  management  and  the 
customer  with  the  major  element 
in  risk  assessment.  Failure 
analysis  means  different  things 
to  different  people,  and 
therefore  a  definition  seems 
appropriate.  Failure  analysis  in 
the  general  sense,  and  as  applied 
through  various  contract 
specifications,  means  the 
determination  of  which  factor 
produced  a  malfunction  and  what 
corrective  action  must  be  taken 
to  eliminate  the  potential  for 
subsequent  failures. 


oxide  step.  Having  identified 
the  point  of  failure  to  this 
level,  the  mechanism  which 
produced  the  failure  must  be 
identified,  i.e.,  the  physics  of 
failure. 

Not  until  the  exact  cause  of 
failure  is  known  can  a  valid 
decision  be  made  regarding  the 
failure  at  the  component  level 
test.  Defining  the  physics  of 
failure  is  also  required  to 
assess  whether  a  piece  part  has  a 
generic  deficiency  which 
could  cause  other  similar  parts 
to  fail,  o£  whether  a  design 
deficiency  exists  which  requires 
redesign  to  eliminate  the 
possibility  of  future  failures. 

What  is  important  about  failure 
analysis  is  that  the  results  must 
provide  the  necessary  information 
to  take  corrective  action  which 
provides  mission  assurance.  The 
depth  of  failure  analysis  and  the 
timely  implementation  of  the 
various  technologies  required  to 
conduct  it  are  mandatory  for 
valid,  objective  management 
decisions  which  assure  the  high 
reliability  required  for 
aerospace  systems. 

FAILURE  ANALYSIS  AT  THE  1978 
MISSION  ASSURANCE  CONFERENCE 


Applying  this  definition,  it  does 
not  suffice  to  say  that  an 
intermittent  "glitch"  occurred, 
without  identifying,  if  at  all 
possible,  exactly  what  caused  the 
glitch;  e. g. ,  test  equipment, 
operator  error,  etc.  Identifying 
the  cause  must  be  validated  by 
duplicating  the  intermittent 
"glitch"  condition,  if  possible, 
and  the  corrective  action  must  go 
beyond  "employee  warned." 

Another  example:  if  analysis  at 
the  component  test  level 
indicates  an  open  condition 
traced  to  an  integrated  circuit 
(IC),  the  IC  must  be  failure 
analyzed  to  identify  where  the 
open  condition  occurred,  e.g.,  a 
broken  wire,  a  lifted  bond,  an 
open  in  the  interconnect 
metallization  on  the  device  at  an 


In  preparation  for  this 
presentation,  a  review  of  the 
1978  Conference  Proceedings  was 
appropriate.  As  a  part  of  this 
review,  it  was  noted  that  the 
keynote  speakers'  presentations 
in  the  first  fourteen  pages  made 
several  references  to  the  impact 
of  mission  failures,  the  need  to 
know  what  causes  failures,  and 
that  the  aerospace  industry  had  a 
"ways  to  go"  relative  to  the 
isolation  of  failure  causes  to  be 
able  to  assess  risks. 

o  Major  General  Howard  E. 
McCormick  -  Vice  Commander  SAMSO 

".  .  .  The  impact  of  a  small 
number  of  failures  can  be 
devastating. 


.  .  Loss  o£  operational 
capability  .  .  .  and  loss  of 
resources. " 

.  .  Impedes  our  ability  to 
obtain  congressional  support 

n 

o  Dr.  John  W.  Townsend,  Jr., 
President,  Fairchild  Space  and 
Electronics  Co. 

"His  (Lt.  Gen.  T.  W.  Morgan) 
challenge  was  to  identify  and 
suggest  solutions  to  general 
problems  that  had  resulted  in  or 
contributed  to  space  mission 
failures. " 

o  Walter  O.  Lowrie, 
Vice-President,  MX  Program, 

Martin  Marietta 

"Cause  of  failure,  personnel 
error  ...  is  a  trap  because  it 
is  a  garbage  can  for  hiding  all 
sorts  of  other  things." 

”.  .  .  treat  all  failures  with 
the  same  rigor  as  if  it  was  a 
flight  failure."  ".  .  .  as  an 
industry,  still  have  a  long  ways 
to  do  better  in  this  concept  of 
cause  isolation  from  the  system 
level  clear  down  to  the  parts 
level  into  the  physics  of 
failure. " 

".  .  .  know  where  the  risks 
specifically  were."  "... 
somebody  at  the  top  level  of  the 
project  personally  assesses  the 
risk  and  buys  off  the  risk." 

In  the  remaining  three-hundred 
pages  of  the  proceedings,  there 
was  virtually  no  reference  to  the 
importance  of  in-depth  failure 
analysis  as  a  management  tool  for 
mission  assurance.  This 
criticism  is  aimed  at 
program/project  management  who 
were  the  presenters  at  the  1978 
Conference.  Below  are  some  of 
the  concepts  abstracted  from 
these  presentations. 

o  We  all  inherently  want  a 
success  -  it's  human  nature 


o  Our  company  must  have  a  good 
reputation  -  new  business 

o  Our  management  provides  in 
depth,  timely  review  of  all 
program  aspects 

o  We  use  a  complex  program 
audit  system  to  assure  program 
success 

o  We  have  numerous,  effective 
program  reviews 

o  Using  independent  readiness 
review  teams  helps,  but  we  need 
more,  sooner 

o  We  want  our  incentive  monies, 
so  we  must  do  a  good  job 

FAILURE  ANALYSIS  IN  THE  1980 
MISSION  ASSURANCE  CONFERENCE 

It  is  obvious  that  the  planners 
of  the  1980  Conference  recognized 
the  need  for  discussions 
concerning  failure  analysis,  and 
elected  that  such  interchange  be 
emphasized  relative  to  the  design 
phase  of  an  aerospace  program. 

The  comments  presented  here 
relate  to  a  high-reliability 
program  where  the  design  is  under 
full  configuration  control  and 
qualification  and/or 
demonstration  system  are  being 
produced  for  delivery.  Whether 
the  program  is  to  produce  a 
commercial  or  government  high 
reliability  spacecraft  is  not 
germane,  the  comments  apply 
equally. 

To  promote  the  desired 
interchange,  speakers  from  two 
government  contractors  and  two 
government  support/consulting 
organization  were  invited  to 
participate.  The  chair  of  this 
session  structured  a  series  of 
questions  that  each  panel  member 
was  to  address.  To  understand 
this  writer's  point  of  view,  the 
following  information  is  offered. 

Since  the  early  sixties, 
assignments  at  two  major 
aerospace  companies  have  involved 
mostly  "fire  fighting"  after 
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failures  have  occurred  at  all 
levels  of  design  and  testing.  No 
matter  how  much  time,  effort  and 
money  were  expended  oversights 
occurred  which  contributed  to 
malfunction.  In  general  the 
aerospace  industry  has  numerous 
exemplary  successes,  but  we  have 
those  few  failures  that  have  such 
a  dramatic  impact  that  success  is 
often  overshadowed. 

With  the  definition  of  design 
phase  and  the  admission  of  being 
a  self  proclaimed  pessimist,  the 
following  comments  are  offered  in 
response  to  the  specific 
questions  of  our  session  chairman. 

WHY  FAILURE  ANALYSIS  IN  THE 
DESIGN  PHASE? 

Most  of  the  reasons  for  in-depth 
failure  analysis  in  the  early 
stages  of  a  program  are  obvious: 

o  Identify  "weak-links"  in  the 
early  stage  of  the  program  so 
corrective  action  has  a  maximum 
reliability  and  minimal  schedule 
impact 

o  Eliminate  the  use  of  less 
than  optimized  parts, 
material  or  processes 

o  Circumvent  delays  and  cost  to 
the  program  caused  by  failures  at 
various  levels  of  assembly  and 
integration 

o  Ambiguous  system/ 
component  performance  during  the 
testing  phase  to  "hard"  reality 
so  valid  engineering  and 
management  decisions  can  be  made 

o  Establish  a  program  baseline 
for  failure  causes  so  trend 
analysis  is  meaningful  and 
possible 

o  Establish  basis  for  critical 
item  control  if  use  of  specific 
parts,  material  or  processes  is 
mandatory  for  fabrication  of  the 
system,  and  is  prone  to  possible 
failure 


Considering  the  above,  it  is 
extremely  important  to  have  an 
understanding  between  the 
contractor  and  customer  to  what 
depth  the  failure  analysis  is  to 
be  conducted  and  just  how  the 
results  will  be  applied  to  the 
program  to  accomplish  the  desired 
mission  assurance  goals.  There 
is  little  doubt  that  the 
contractors  commitment  must  be 
tempered  with  a  "cost  effective" 
factor;  however,  the  in-depth 
analysis  is  a  prerequisite  to  the 
determination  of  what  is  cost 
effective. 

recommendation ; 

Considering  the  question,  it 
seems  obvious  that  the  customer 
should  define  requirements 
explicitly  in  the  RFP,  and  that 
each  potential  contractor  should 
respond  in  an  explicit  manner  so 
that  an  objective  comparison  can 
be  made  a  detail  part  of  source 
selection. 

WHERE  DOES  THE  RESPONSIBILITY  FOR 
FAILURE  ANALYSIS  REST  IN  A 
PROJECT  OFFICE? 

It  is  virtually  impossible  to 
provide  a  general  answer  to  this 
question  for  all  types  of 
contracts;  each  contract  must 
exercise  options  to  place  the 
responsibility  in  an  independent, 
objective  quality  related 
function,  but  must  also  assure 
that  project  office  obligations 
and  responsibilities  are  met. 

Each  contractor's  functional 
management  should  establish  by 
policy/procedure  the  requirement 
for  the  quality  function  to  be 
responsible  so  program/project 
shortcuts  in  failure  analysis  are 
not  allowed  to  occur.  With  these 
considerations  in  mind,  the 
following  concept  is  offered: 

o  The  responsibility  must  be 
shared  between  the  program 
manager  and  the  quality  assurance 
(Reliability/System 
Effectiveness)  manager 
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o  The  customer  must  be  notified 
of  all  failures  at  all  levels. 

o  The  depth  of  analysis  must  be 
defined  by  QA  with  program  office 
understanding. 

o  The  customer  must  concur  with 
depth  of  analysis  defined. 

o  The  program  and  QA  manager 
must  evaluate  failure  analysis 
results  and  concur  with 
recommended  corrective  action. 

o  The  failure  must  be  closed 
out  through  the  Failure  Review 
Board  with  customer  right  of 
disapproval. 

o  Corrective  actions  must  be 
implemented  through  Corrective 
Action  Board  with  customer 
concurrence. 

The  above  concept  assumes  that 
the  "human  equation  factors"  of 
team  effort  are  realizable  and 
that  adequate  personnel, 
facilities  and  funding  are 
available;  wonderful  assumptions 
that  are  difficult  to  attain,  but 
which  must  be  pursued  with 
maximum  vigor. 

RECOMMENDATION; 

After  failure,  an  objective 
quality  organization  should  have 
primary  responsibility  to 
determine  the  level  of  analysis 
to  be  conducted  and  should  define 
the  corrective  actions  to  be 
implemented.  A  major 
consideration  would  be  the 
company's  policy,  established  by 
company  functional  management, 
that  assures  that  program/project 
management  does  not  overpower  the 
quality  organizations  charter  to 
control  failure  analysis 
activities. 

The  customer  should  provide 
technical  assistance  and  reserve 
the  right  of  disapproval. 


WHAT  IS  THE  VALUE  OF  FAILURE 
ANALYSIS  TO  PROGRAM  SUCCESS  AND 
SCHEDULE? 

This  question  is  very  difficult 
to  answer,  it  depends  where  you 
are  "coming  from".  Failure  prone 
materials  and  parts  which  are  not 
detected  early  in  the  program, 
essentially  guarantee  that  the 
program  will  not  meet  cost, 
performance  and  schedule 
objectives.  The  failures  will 
happen.  From  the  writers  view: 

o  Each  failure  is  indicative  of 
a  potential  schedule  problem 
and/or  mission  failure 

o  Early  failures  will  allow  for 
redesign,  reselection  of  parts, 
application  of  enhanced  screening 
and  testing,  or  other  practical 
corrective  actions 

o  Comprehensive  in-depth 
failure  analysis  allows  for 
objective  risk  assessment  and 
program  management/customer 
decisions. 

The  three  above  factors  value  to 
a  program's  success  can  best  be 
illustrated  by  examples: 

o  In  an  early  ICBM  program 
(early  1960s),  a  major  supplier's 
integrated  circuits  (ICs) 
experienced  numerous  failures  at 
the  component  level.  At  that 
time  there  were  reportedly  50,000 
ICs  on  the  shelf  at  the 
contractors  and  50,000  ICs  in  the 
"pipe-line"  at  the  suppliers. 
After  a  comprehensive  failure 
analysis,  the  time  temperature 
mechanism  was  defined  and  an 
unique  electrical  screen 
developed  which  allowed  the 
program  to  go  forward  using 
rescreened  ICs. 

o  In  an  ongoing  military 
satellite  program,  numerous 
failures  were  experienced  with  a 
DTL  integrated  circuit.  Failure 
analysis  indicated  that  gold 
bail-bonds  were  prone  to  failure 
for  two  reasons:  1) 
metallization  was  thin  at  the 


bond  pad  and  the  package  was 
furnace  solder  sealed.  The 
combined  situation  was  shown  to 
have  a  high  probability  of 
failure,  therefore,  the  units 
were  retrof i tted. 

o  In  a  more  recent 
communication  satellite  program, 
a  launch  was  delayed  because  two 
identical  satellites  in  assembly 
and  test  had  experienced  several 
transistor  failures.  Detail 
failure  analysis  identified  the 
time- temperature  dependent  wire 
bond  failure  mechanism  and  also 
identified  that  there  was  bond 
redundancy  within  the  power 
transistor.  Considering  the  fact 
that  failures  were  being  caught 
during  system/component  tests  and 
the  launch  satellite  had  not 
experienced  failure,  and  that 
redundancy  within  the  transistor 
would  require  two  failures  to 
cause  device  failure,  and  that 
the  satellite  had  redundant 
components,  the  launch  was 
authorized  and  the  satellite  has 
over  3  years  of  successful 
orbital  operation.  Of  course, 
all  transistors  in  the  two 
satellites  in  assembly  and  system 
test  were  retrofitted. 

o  A  recent  government  satellite 
program  had  a  continuing  series 
of  Central  Processing  Unit 
failures  traced  to  a  hybrid  RAM. 
After  extensive  failure  analysis 
by  the  hybrid  supplier  and 
numerous  "finger-pointing" 
meetings  between  the  device 
supplier,  the  hybrid  producer  and 
the  system  contractor,  the  device 
supplier  was  indicted.  As  a 
result,  the  supplier  "cleaned  up 
his  act",  the  hybrid  producer 
performed  more  extensive 
electrical  screening  of  the  RAM 
device  before  hybrid  assembly  and 
of  the  hybrid.  The  contractor 
performed  more  extensive  testing 
at  the  printed  wiring  board  and 
component  level.  The  customer 
also  established  the  capability 
to  simulate  on  orbit  malfunction 
in  the  event  they  occurred,  and 
the  ability  to  reprogram  the  CPU 


to  reestablish  a  failed  CPUs 
functionality. 

It  is  hoped  that  these  examples 
indicate  the  significance  of 
failure  analysis  relative  to  risk 
assessments  made  early  in  the 
program,  until  launch,  and 
after.  In-depth  failure  analysis 
reduces  these  risk  assessments  to 
engineering  decisions  based  on 
facts  and  data,  rather  than 
decisions  based  on  "gut-feelings" 
and  program  pressures. 

RECOMMENDATION ; 

Use  failure  analysis  early  in  the 
program  to  detect  and  define 
problems  so  cost,  performance, 
and  schedule  objectives  can  be 
met.  Use  in-depth  failure 
analysis  to  both  circumvent 
problems  which  could  occur,  and 
to  expeditiously  solve  those 
which  do  occur. 

HOW  IS  LASTING  CORRECTIVE  ACTION 
ASSURED? 

When  a  failure  occurs,  and  the 
necessary  analysis  has  determined 
the  physics  which  caused  the 
malfunction,  both  supplier  of  a 
material  or  piece  part,  and/or 
the  contractor  are  implicated. 
Someone  inadvertently  or 
consciously  "slipped-up. "  The 
customer  could  also  be 
implicated.  In  any  case,  they 
are  all  in  it  together  so  the 
necessary  corrective  actions 
should  be  determined,  concurred 
with  and  implemented  as 
expeditiously  as  possible. 

Voluntary  corrective  action  is  a 
rare  event  when  you  experience 
it;  "I  made  a  mistake  and  I'll 
correct  it,"  are  words  seldom 
heard.  If  such  is  the  case,  and 
the  corrective  action  is 
technically  and  economically 
valid,  the  job  is  done. 

But,  more  generally  it  becomes  a 
real  fight: 

o  We  met  the  spec,  it  is  not 
our  problem  that  it  failed. 


o  We  conducted  an  extensive, 
objective  series  of  tests  that 
shows  that  failure  was  one  in  a 
million.  Data  you  ask?  Don't 
have  that  right  now  bnt  -  . 

o  Correction  action?  We'll 
take  care  of  that  if  we  get  a 
follow-on  contract,  or  additional 
funding. 

o  This  is  a  one-shot  satellite 
program  and  corrective  action 
requirements  were  not  in  the 
contract. 

o  This  part  has  never  failed 
before.  How  many  programs  have 
used  it  before?  I'll  have  to 
look  into  that. 

o  We  make  it  the  way  we  always 
have  and  if  it  failed,  you  must 
have  done  something  wrong. 

Quality  Control  you  say;  what's 
that? 

With  this  type  of  response,  the 
battle  lines  are  drawn  and  under 
these  conditions  the  contractor, 
customer  or  government  must 
attempt  to  force  corrective 
action  through  new  specifica¬ 
tions,  a  new  supplier,  unique 
contracting,  etc.  The  government 
often  mandates  it  through 
contract  requirements  and 
specifications  but  again 
generally  not  without  a  fight. 

The  significance  is  that  delayed 
corrective  actions  is  potentially 
costly  in  dollar,  schedule  and 
performance. 

COMMENT; 

Most  corrective  action  is  done  on 
a  case-by-case  basis  and  it  is 
difficult  to  generalize  relative 
to  the  effectiveness.  However 
since  reliability  and  cost 
effectiveness  are  paramount, 
perhaps  an  award  fee  type  of 
arrangement  for  voluntary 
implementation  of  significant 
corrective  actions  would  be 
warranted,  especially  in  the 
design  phase.  The  main  concern 
is  that  lessons  learned  are  not 
forgotten  or  overlooked. 


Obviously,  voluntary  corrective 
action  is  more  timely  than  forced 
corrective  action.  Another 
consideration,  numerous 
operational  failures  have  been 
identified  to  known  failure 
mechanisms,  which  were  ignored 
and  no  corrective  action  taken. 

HOW  BEST  'X>  EXCHANGE  FAILURE 
ANALYSIS  INFORMATION? 

For  those  who  have  worked  in  the 
failure  analysis  technology  field 
for  a  number  of  years,  the  best 
answer  to  this  is  -  get  on  the 
phone  and  call  one  of  your 
colleagues  who  specializes  in  the 
technical  area  of  concern.  There 
is  no  better  way,  because  the 
co-workers  are  concurrently 
experiencing  the  same  problem,  or 
have  already  progressed  to  the 
point  of  implementing  corrective 
action.  Another  thing,  be  sure 
that  there  is  a  mutual  exchange, 
not  just  enough  biased 
interchange  to  support  your  or 
your  boss's  theory.  "Bounce" 
your  information  off  your 
customer;  he  may  have  a  lead. 
Telecon  or  direct  interface  with 
your  fellow  technologist  has 
started  many  analyses  off  on  the 
"right-foot"  and  had  a  very 
positive  result  in  cost  avoidance 
and  timely  solutions  to 
problems.  Also,  don't  stop  with 
the  first  expert;  you  need  a 
consensus,  if  possible.  Another 
point,  these  verbal  interchanges, 
with  some  degree  of  discretion, 
should  never  result  in  adverse 
legal  action  against  your 
employer. 

The  other  presentations  in  this 
session  will  detail  how  their 
companies  or  organizations 
provide  for  the  interchange  of 
failure  analysis  information,  and 
most  probably  their  systems  are 
satisfactory  to  their  customers. 
The  real  question  is  how 
effective  is  the  system  and  is 
the  intent  of  such  a  system 
understood. 

Within  a  company,  unless  leaks 
occur,  a  supplier's  product  or 
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procedure  can  be  "bad-mouthed" 
without  too  much  concern  for 
legal  action.  Therefore,  you  can 
call  your  shots  as  you  see  them 
and  other  programs  can 
effectively  use,  if  they  are 
astute,  the  information  to 
correct  or  circumvent  problems. 
The  real  concern,  especially 
within  a  large  company,  is 
getting  the  principals  involved 
in  the  failure  analysis  to  write 
the  problem  and  solution  into  the 
system,  preferably  from  the 
occurrence  of  failure, 
periodically  updating  and  finally 
detailing  the  mechanism  and 
corrective  action.  Assuming  that 
this  information  is  written  into 
the  systems,  does  each  program  in 
the  company  then  draw  from  that 
system?  Do  the  individuals 
involved  in  reviewing  the 
system's  information  recognize 
"their"  problems  from  the 
description  provided?  Does  the 
description  of  the  problem 
readily  delineate  generic  facts 
that  should  allow  a  proper 
extension  of  the  concern  to  all 
possible  products?  Physics  of 
failure  is  based  on  the  laws  of 
nature,  not  on  a  specific 
materials  or  material  combination 
or  a  lot  date  code,  or 
necessarily  one  supplier's 
product.  This  fact  must  be 
understood  and  employed  while 
using  the  companies  failure 
analysis  reporting  system. 

Again,  if  the  human  factor  and 
technical  experience  is  present, 
and  the  system  works  as  it 
should,  most,  if  not  all  known 
failures  can  be  minimized  or 
totally  circumvented.  Some  more 
wishful  thinking  - 

Technical  conferences  are  a  blend 
between  the  person-to-person 
interface  when  experts  meet  and 
talk  to  each  other,  and  the 
company  systems  where  written 
technical  reports  are  presented. 
Of  course,  the  technical 
presentations  at  a  conference 
provide  a  higher  risk  relative  to 
legal  recourse.  Nonetheless, 
these  conferences  are  a  necessary 
element  in  the  contining  effort 


to  expand  the  technical  base  for 
prevention  of  failures,  or  if 
they  occur,  to  their  solution. 

It  is  the  writer's  opinion  that 
the  Government-Industry  Data 
Exchange  Prograun  (GIDEP)  Failure 
Alert  system  does  not  yet 
accomplish  what  is  needed.  Most 
certainly  there  are  cases  where 
GIDEP  has  served  its  intended 
function,  and  prevented  program 
delays  and  failures.  However, 
all  too  often  valuable 
information  is  not  inputed  into 
GIDEP  at  all,  or  is  "watered 
down"  to  the  degree  it  is 
ineffective.  Often  there  is  an 
exceedingly  long  delay  between  a 
failure  definition  and  corrective 
action  delineation,  and  the  input 
into  the  GIDEP  Alert  system. 
Basicly,  these  are  related  to 
personnel  and/or  legal  problems. 

RECOMMENDATION; 

Government  and  industry  must 
foster  the  exchange  of 
comprehensive  failure  analysis 
information  by  promoting  existing 
programs,  and  conceiving  and 
implementing  innovative  real  time 
concepts.  Legal  recourse 
coercion  aspects  must  be 
investigated,  and  potential 
adverse  liability  factors 
minimized.  Personnel  involved  in 
data  exchange  efforts  must  be 
trained  to  understand  failure 
analysis  technology  and  to 
incorporate  this  knowledge  into 
the  preparation  of  inputs.  Cases 
where  suppliers  have  refused  to 
provide  product  as  retaliation 
for  adverse  reports  must  also  be 
considered,  especially  in  the 
current  economic  conditions  where 
commercial  sales  are  more 
important  than  military  sales. 

HOW  DO  YOU  PROFIT  FROM  FAILURE 
ANALYSIS  RESULTS  IN  CURRENT  AND 
FUTURE  PROGRAMS? 

This  paper  has  certainly  touched 
on  this  question  in  the  previous 
paragraphs  but  perhaps  a 
reiteration  at  this  juncture 
would  be  beneficial: 
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o  Properly  extend  the  results 
of  failure  analysis  to  other  lot 
date  codes  and  generic  part  type 
and  materials 

o  Assure  that  corrective 
actions  are  incorporated  into 
program  used  materials  and  parts 

o  Incorporate  your  knowledge  of 
potential  failure  causes,  and 
lessons  learned  into  supplier 
baseline,  source  control  drawing 
requirements,  destructive 
physical  analysis  documents,  and 
incoming  inspect ion/ test 
requirements 

o  Let  industry  know  the 
problems  and  solutions,  and 
hopefully  they  will  reciprocate 

o  Attend  failure  analysis  and 
reliability  conferences  and 
participate  in 

industry/government  sponsored 
activities 

o  Look  to  the  past  for  lessons 
learned  to  assure  they  have  not 
been  forgotten 

RECOMMENDATION; 

Assure  that  a  closed  loop  review 
procedure  which  maintains  a 
continual  monitoring  of  failure 
analysis  and  corrective 
actions  has  been  established  and 
maintained  relative  to  current 
and  future  program  needs.  Review 
your  company's  and  industry's 
"historical  files"  at  program 
inception  to  assure  that 
maximized  benefits  are  derived. 

ARE  CUSTOMER  SPECIFICATIONS, 
REQUIREMENTS  AND  WORK  STATEMENT 
ADEQUATE? 

It  is  about  time  that  we  consider 
what  may  be  the  most  limiting 
factor  in  using  failure  analysis 
technology  for  maximum  benefit  to 
mission  assurance.  The  various 
documents  in  a  RFP  package 
attempt  to  identify  to  the 
contractor  what  is  required  for 
meaningful  failure  analysis  to 
support  mission  success.  In 


general,  the  contractors  response 
is,  "we  will  comply."  At  this 
point,  there  most  likely  is  not 
a  meeting  of  the  mind  exactly 
what  is  desired,  and  what  the 
bidder  has  committed.  Who  can 
predict  how  many  failures  will 
occur;  to  what  extent,  will  the 
analysis  be  necessary,  (how 
expensive)?  How  large  an  impact 
will  the  resulting  corrective 
actions  have  on  the  program? 

RECOMMENDATION; 

The  following  simple  statements 
are  offered  as  a  suggested 
solution  to  this  dilemma; 

o  The  customer  must  explicitly 
define  what  is  required 
in  the  RFP. 

o  The  contractor  must 
explicitly  define  what  his 
perceived  obligations  are  in 
response  to  the  RFP 

o  Final  negotiations  between 
customer  and  contractor  must 
reconcile  any  differences. 

o  The  contractor  must  then 
define  explicitly  his  obligation, 
and  cost  the  failure  analysis 
effort  into  his  proposal.  The 
costs  should  be  a  separate  and 
distinct  element  of  his  proposal. 

o  The  customer  must  fund  at  the 
anticipated  level  to  assure  that 
failure  analysis  requirements  are 
met,  and  must  be  prepared  to 
augment  these  funds  if  the 
customer  can  justify  that 
extenuating  circumstances,  not 
under  his  control,  have 
precipitated  the  need.. 

Having  made  these  simplistic 
statements,  accomplishing  their 
intent  then  becomes  a  major 
problem.  Each  program  must 
address  the  requirements  relative 
to  their  goals.  Unfortunately, 
available  monies  are  generally 
the  driving  force  which  limits 
the  implementation  of  effective 
failure  analysis.  Also  consider, 
a  contractor  with  several  hi-rel 


programs  using  the  same 
materials,  parts  and  procedures, 
may  have  a  competitive  advantage 
when  a  failure  analyses  is 
required,  i.e.  cost  can  be  shared. 

Contractor  and  customer  must 
negotiate  explicit  failure 
analysis  requirements  and 
commitments  for  each  problem,  and 
funding  must  be  made  available  to 
accomplish  the  effort. 

HOW  DO  YOU  ASSURE  EFFECTIVE  FMEA 
AND  CONTROL  OF  SINGLE  POINT 
FAILURES  (CRITICAL  ITEMS)? 

This  question  of  Failure  Mode  and 
Effects  Analysis  (FMEA)  relative 
to  failure  analysis  performance 
seems  somewhat  moot.  However, 
the  point  can  be  made  that  any 
parts,  materials  or  processes 
related  to  a  singe-point-failure 
adds  significance  to  the  extent 
of  failure  analysis  to  be 
performed  in  the  event  a 
malfunction  occurs.  Conversely, 
if  failure  analysis  data  exists 
which  indicates  that  a 
single-point-failure  part  or 
material  has  a  higher  probability 
to  fail  by  a  known  mechanism  than 
is  "acceptable,"  redesign  can  be 
mandated,  or  special  controls 
and/or  screens  can  be  implemented 
to  reduce  the  possibility  to  an 
"acceptable"  level.  It  should  be 
emphasized  that  it  could  be  very 
dangerous  to  make  assumptions  as 
to  the  depth  of  failure  analysis 
activity  relative  to  the 
consequence  of  failure  as 
determined  by  FMEA.  This  type  of 
assumption  could  overlook  the 
significance  of  similar  failure 
mechanisms,  and  restrict  the 
latitude  of  redundant  design. 

With  the  above  in  mind,  the 
following  comments  are  presented: 

o  Each  contractor  shall  conduct 
FMEA  to  define  program 
imperative"  single-point- 
failures  (SPF).  Program 
imperative  implies  that  the 
system  cannot  be  built  without 
the  SPF. 


o  Each  designer  must  assume 
responsibility  for  FMEA  of  his 
design  and  objectively  assess 
that  a  SPF  is,  in  fact,  program 
imperative. 

o  The  reliability  organization 
should  verify  FMEA  results  as 
early  in  the  program  as  possible 
to  validate  program  imperative 
SPF. 

o  All  parts,  materials  and 
processes  (PM&P)  related  to  each 
single-point  failure 
automatically  becomes  a  "critical 
item"  requiring  a  special 
attention  plan  defining  special 
processing,  and  application 
requirements. 

o  Failure  analysis  and  DPA  of 
all  PMSrP  associated  with  a 
critical  item  shall  require  a 
maximum  level  of  effort  (define 
potential  or  known  failure 
mechanism,  establish  special 
screens  to  eliminate  such 
mechanisms  or  implement  validated 
corrective  action  to  eliminate 
the  mechanism). 

o  Use  failure  analysis 
information  as  a  tool  to  more 
thoroughly  assess  the  FMEA 
assumptions  (e.g,  shorted  diode, 
open- transistor )  relative  to  the 
probability  of  failure  occurrence 
and  the  impact  on  system 
vulnerability. 

RECOMMENDATION; 

All  RFPs,  and  responses  to  RFP 
should  define,  in  detail  the 
program  requirement  for  FMEA,  and 
explicit  details,  responsibility 
and  procedures  relative  to 
program  imperative  single-point 
failures  and  critical  item 
controls . 
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MISSION  ASSURANCE  CONFERENCE 


DESIGN  WORKSHOP 

CONFIGURATION  ITEM  SPECIFICATIONS 

L.A,  Hartman 
Lockheed,  LMSC 
Ernie  Wade 

The  Aerospace  Corporation 

AGENDA: 

In  this  part  of  the  besign  Workshop,  A1 
Hartman  will  make  a  brief  presentation 
stating  why  we  are  including  configuration 
item  specification  formats  and  contents 
in  a  conference  aimed  at  enhancing  assur¬ 
ance  of  mission  success,  some  background 
data  and  a  number  of  issues  to  which  we 
desire  your  opinion.  The  issues  will 
be  presented  in  total  and  we  will  then 
address  each  one  within  the  group  dis¬ 
cussion  period.  Therefore,  we  request 
that  you  refrain  from  asking  questions 
during  the  presentation.  After  disposing 
of  the  presented  issues,  any  other  issue 
with  respect  to  preparation  of  specifi¬ 
cations  may  be  advanced  for  discussion. 

Ernie  Wade  will  then  summarize  the  group 
opinion  from  which  the  recommendations 
will  be  derived  for  presentation  to  the 
conference  leaders. _ 

AGENDA 

•WHY  Cl  SPECS’ 

•  REQUIREMENTS  BACKGROUND  AL  HARTMAN  LMSC 

•  THE  ISSUES 

•  GROUP  DISCUSSION  ALL  ATTENDEES 

•  SUMMARY  ERNIEWADE  AEROSPACE 

WHY  SPECS? 

Configuration  item  specifications  can  and 
frequently  do  divert  resources  from  those 
activities  that  directly  support  assurance 
of  achieving  mission  objectives.  The 
misapplication  of  resources  occurs  when¬ 
ever  excessive  or  gold  plating  require¬ 
ments  are  included  in  the  specification 
without  regard  to  any  cost  benefit  analysis. 
Confusing  or  ambiguous  requirements  always 
waste  resources.  Incomplete  require¬ 
ments  have  great  potential  for  causing 
mission  failure  in  performance,  cost,  or 
schedules.  Therefore,  our  goal  is  to 
achieve  a  set  of  specifications  that 


accurately  reflect  true  mission  require¬ 
ments  in  order  to  ensure  that  resources 
are  only  expended  that  assure  mission 
success. 

Current  practice  with  respect  to  specified 
requirements  is  that  all  requirements  are 
of  equal  importance  and,  therefore,  all 
requirements  must  be  fulfilled.  There  is 
an  order  of  precedence  implied  by  the 
tiering  within  the  specification  tree  in 
the  event  of  conflict  between  levels  of 
specification.  However,  resources  can  be 
wasted  trying  to  achieve  fulfillment  of 
a  functional  requirement  which  would  have 
a  minor  effect  on  achieving  mission  goals. 
The  order  of  precedence  must  be  clearly 
stated  to  prevent  such  diversion  of  effort. 

Another  diversion  to  assuring  mission 
success  is  the  tendency  to  spend  excessive 
resources  in  the  preparation  and  review  of 
the  specifications.  A  lot  of  these  effects 
are  hidden.  Costs  are  hidden  since  not 
all  costs  show  up  as  "Specification  Pre¬ 
paration"  costs.  No  one  summarizes  all 
of  the  design  review  costs  as  they  are 
scattered  over  many  unrelated  organizations. 
Schedules  reflect  desires  but  are  seldom 
achieved.  We  are  each  aware  of  the  many 
manhours  going  into  specification  pre¬ 
paration  and  review  and  the  failure  in 
having  approved  specifications  for  the 
next  phase. 

Last  but  not  least,  we  are  examining  spec¬ 
ifications  in  this  workshop  since  they  are 
the  media  used  for  documenting  primary 
and  derived  mission  requirements.  Such 
documentation  has  the  potential  for 
enhancing  or  distracting  efforts  leading 
to  the  achievement  of  a  successful  mission. 

A  specification  which  supports  mission 
assurance  includes  only  needed  information, 
clearly  stated,  leaving  no  room  for  gues- 
sing  what  is  wanted. _ 

WHY  SPECS? 

•  FAILURE  TO  LIMIT  REQUIREMENTS  TO  NEEDS  RESULTING 
IN  MISAPPLICATION  OF  RESOURCES 

EXCESSIVE  REQUIREMENTS 

CONFUSING  OR  AMBIGUOUS  REQUIREMENTS 

INCOMPLETE  REQUIREMENTS 

•  ORDER  OF  PRECEDENCE  OF  REQUIREMENTS  IS  NOT 
CLEARLY  STATED 

•  EXCESSIVE  PREPARATION  AND  REVIEW  DIVERTING 
RESOURCES  FROM  FULFILLING  MISSION  OBJECTIVES 

•  SPECIFICATIONS  DOCUMENT  MISSION  REQUIREMENTS  AS 
RECOGNIZED  8  DERIVED 
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BACKGROUND: 


ISSUES: 


The  Air  Force,  in  the  first  configuration 
management  manual,  defined  a  uniform 
specification  program  to  replace  the  more 
generalized  Defense  Standardization  Manual 
M200B  of  1  April  1966  for  application  to 
Air  Force  programs.  Three  types  of  spec¬ 
ifications  were  identified;  Detail, 
Identification  and  Requirement  Specifi¬ 
cations.  That  issue  of  375-1  was  quickly 
followed  by  the  1964  issue  which  defined 
new  types  of  specifications  to  replace 
the  former  types.  They  were  the  System 
General,  the  Contract  End  Item  (CEI) 

Prime  Equipment,  CEI  Facility,  CEI  Ident¬ 
ification  Item,  CEI  Requirements  Items, 
Critical  Components  and  Company  Standard 
Parts. 

At  that  point,  the  DOD  suggested  that  a 
uniform  approach  be  adopted  across  all 
components  and  MIL-STD-490  was  the  result 
of  total  DOD  coordination.  Thus  490  tried 
to  cover  specifications  for  all  military 
items  produced  to  contractor  prepared 
specifications;  running  the  full  breadth 
from  ships  to  rifle  bullets.  M200B  was 
reissued  as  DOD  4120. 3-M  to  govern  MIL 
specifications  and  standards  prepared  by 
the  Government  or  by  contractors  for  the 
Government. 

Since  MIL-STD-490  is  a  compromise  document 
representing  some  requirements  not  fully 
endorsed  by  the  Air  Force,  MIL-STD-483 
(USAF)  was  issued  which  directs  certain 
usages  of  options  permitted  by  490  and 
adds  additional  requirements  to  490. 
However,  MIL-STD-483  is  also  a  compromise 
document  since  all  Air  Force  procurement 
is  covered.  Therefore,  there  is  no  spec¬ 
ification  preparation  manual  that  is 
directed  to  the  uniqueness  of  space 
programs. 

BACKGROUND 

•  AFSCM  375  1  I  JUNE  1962.  EXHIBIT  II,  UNIFORM 
SPECIFICATION  PROGRAM 


•  AFSCM  3751,  1  JUNE  1964  EXHIBIT  I  II  III  IV  AND  V 


•  MIL  STD  490  30  OCTOBER  1968  APPLICABLE  TO  ALL  DOD 
COMPONENTS 


•  MIL  STD-483  (USAF)  31  DECEMBER  1970 


With  that  background,  we  workshop  leaders 
put  together  some  propositions  in  order 
to  obtain  your  reaction.  We  have  no 
intention  of  limiting  the  discussion 
period  to  consideration  of  only  these 
issues.  If  any  one  cares  to  submit 
another  proposition,  use  the  cards  pro¬ 
vided.  We  do  want  you  to  wait  until  we 
have  disposed  of  the  ones  I  present  here. 
We  will  run  thru  all  the  issues  and  then 
discuss  them  one  at  a  time. 


ISSUES 


ISSUE  1: 

The  Part  II,  Product  Specifications,  are 
not  approved  until  Physical  Configuration 
Audit  (PCA)  which  comes  at  the  end  of  the 
development  phase.  They  are  intended  to 
be  the  contractual  instrument  for  a 
production  phase.  Since  the  majority  of 
our  space  program  procurements  do  not  have 
a  production  phase,  why  not  eliminate  the 
requirement  for  these  additional  docu¬ 
ments  by  including  their  unique  material 
directly  within  the  Part  I  specifications? 
PCA  can  be  conducted  as  an  audit  against 
the  Part  I  spec's  fabrication  and 
acceptance  requirements. 

ISSUE  1 

IF  THERE  IS  TO  BE  NO  PRODUCTION  PHASE,  SHOULD 
PART  II  SPECIFICATIONS  (MIL  STD-490,  TYPE  C)  BE 
ELIMINATED  BY  DOCUMENTING  FABRICATION  AND 
ACCEPTANCE  REQUIREMENTS  IN  THE  DEVELOPMENT 
(PART  I)  SPECIFICATION? 

ISSUE  2: 

The  first  part  of  Issue  2  is  the  same  as 
Issue  1  as  applied  to  software  specifi¬ 
cations.  There  are  two  more  distinctions 
between  hardware  and  software  specifi¬ 
cations  which  should  be  addressed.  Since 
MIL-STDs-490/483  only  portray  a  Computer 
Program  Configuration  Item  (CPCI) 
specification,  there  is  a  reaction  that 
requirements  at  all  levels  with  the  CPCI 
be  documented  in  a  single  specification. 
The  question  is,  why  not  provide  for  lower 


ISSUE  3 


level  specifications  such  as  at  module  or 
routine  levels?  A  third  consideration  is 
that  current  practice  requires  that  the 
as-built  listings  be  incorporated  into 
the  specification  resulting  in  a  huge 
document  if  the  listings  are  extensive. 
Another  problem  occurs  when  there  are  two 
or  more  copies  of  the  CPCI  in  use  within 
multiple  stations  all  of  just  slightly 
different  versions,  requiring  multiple 
listings  in  the  specification.  Therefore, 
we  propose  that  the  listings  be  attached 
to  or  accompany  the  VDD  for  each  copy 
of  the  program. 


ISSUE  2 

WHY  NOT  TREAT  SOFTWARE  SPECIFICATIONS  THE 
SAME  AS  HARDWARE  SPECIFICATIONS  BY  (1)  INCLUDING 
CONSTRUCTION  AND  ACCEPTANCE  REQUIREMENTS  IN  THE 
DEVELOPMENT  (PART  I)  SPECIFICATION  (WHICH  ELIMINATES) 
THE  NEED  FOR  A  PRODUCT  (PART  II)  SPECIFICATION  (REF 
ISSUE  I),  (2)  PROVIDING  FOR  SPECIFICATION  LEVELS 
SIMILAR  TO  PRIME  ITEM,  CRITICAL  ITEM  AND  NON 
COMPLEX  ITEM  SPECIFICATIONS  AND  (3)  PLACING 
THE  AS  BUILT  ■  LISTINGS  AS  AN  APPENDIX  TO  THE 
VERSION  DESCRIPTION  DOCUMENT  (VDD)’ 


ISSUE  3: 

In  addition  to  Issues  1  and  2,  there  are 
other  reasons  for  including  fabrication 
requirements  in  development  specifications 
and  also  in  system  specifications.  When 
the  physical  interface  is  now  due  to 
utilization  of  existing  equipment,  the 
interface  should  be  specified  in  the  A 
or  B  specification.  A  new  application  of 
existing  hardware  in  original  or  modified 
configuration  can  be  specified  in  place 
of  specifying  development  requirements. 
Development  is  accomplished  over  time  and 
the  development  specification  could  be 
used  to  document  progress  to  guide  further 
development  effort.  Also,  the  specification 
could  document  lessons  learned  to  guide 
future  application  of  the  item  or  similar 
i terns. 


WHY  NOT  PERMIT  INCORPORATION  OF  BOTH  FUNCTIONAL 
AND  FABRICATION  REQUIREMENTS  IN  TYPE  A  AND  B 
SPECIFICATIONS  PARTICULARLY  TO 

•  IDENTIFY  PHYSICAL  INTERFACES’ 

•  IDENTIFY  NEW  APPLICATIONS  OF  EXISTING  SPACE  S 
SUPPORT  ITEMS’ 

•  DOCUMENT  THE  DEVELOPMENT  AS  IT  PROCEDES’ 

•  DOCUMENT  LESSONS  LEARNED’ 


ISSUE  4: 

After  the  specification  has  been  approved 
to  establish  a  configuration  baseline, 

DOD  MIL-STD-480' s  definition  of  a  class  I 
change  demands  that  all  future  changes 
to  the  specification  be  processed  as 
formal  engineering  change  demands  that 
all  future  changes  to  the  specification 
be  processed  as  formal  engineering  change 
proposal  (ECP)  submittals.  Why  must  all 
changes  take  this  route?  There  is  a  lower 
level  of  approval  permitted  for  baseline 
drawing  changes  so  why  mot  use  it  for 
minor  changes  to  the  specification  such 
as  the  four  identified  on  this  illustra¬ 
tion?  Too  often  a  corrective  change  is 
left  undone  due  to  the  high  cost  of  pro¬ 
cessing  a  formal  ECP. 


ISSUE  4 

AFTER  SPECIFICATION  APPROVAL  DOD  STD-480A  REQUIRES 
SUBMISSION  AND  APPROVAL  OF  A  CLASS  I  CHANGE  PRIOR 
TO  CHANGING  THE  SPECIFICATION  S  CONTENT  IN  ANY 
MANNER.  WHY  NOT  USE  CLASS  II  CHANGES  APPROVED  BY 
THE  CUSTOMER  S  REPRESENTATIVE  (AFPRO  DCAS)  FOR 

•  UPDATE  OF  SECTION  2  APPLICABLE  DOCUMENTS  TO  RECORD 
CURRENT  DOCUMENT  REVISIONS’ 

•  CALLOUT  OF  CONTRACTOR  REPLACEMENT  PART  NUMBERS’ 

•  ERROR  CORRECTIONS’ 

•  ADD  INFORMATION  AS  SUGGESTED  BY  ISSUES  I  AND  2’ 


ISSUE  5: 

As  lower  level  specifications  are  generated 
and  baseline  approved,  the  summation  of 
the  inmediate  lower  level  specif ication(s) 
contain(s)  the  full  set  of  requirements 
from  the  parent  documents.  The  parent 
document  is  in  effect  redundant  to  the 
lower  level  documents.  Continued  main¬ 
tenance  of  all  levels  of  specification 
will,  therefore,  require  effort  of  a 
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redundant  nature.  Why  not  discontinue  main¬ 
tenance  of  the  higher  level  specifications 
when,  in  effect,  they  have  been  totally  re¬ 
placed  by  the  lower  level  set  of  specifica¬ 
tions.  If  a  major  change  requires  addition¬ 
al  development  effort,  then  the  higher  level 
specifications  for  the  modification  should 
probably  be  updated  or  a  new  set  of  develop¬ 
ment  specifications  generated.  Resources 
saved  can  be  applied  elsewhere  to  promote 
the  achievement  of  mission  assurance. 

ISSUE  5 

MIL  STDs  483  AND  490  IMPOSE  REQUIREMENTS  FOR 
MAINTAINING  ALL  SPECIFICATIONS  AT  ALL  LEVELS 
THROUGHOUT  THE  SYSTEM  S  LIFE  CYCLE.  WHY  NOT 
DISCONTINUE  SPECIFICATION  MAINTENANCE  IF  THE 
SPECIFICATION  HAS  BEEN  EFFECTIVELY  REPLACED  BY 
LOWER  LEVEL  SPECIFICATIONS’ 

•  DISCONTINUE  TYPE  A  MAINTENANCE  WHEN  ALL  TYPE  Bs 
HAVE  BEEN  APPROVED 

•  DISCONTINUE  TYPE  B  PART  I  MAINTENANCE  WHEN  ITS  TYPE 
C  PART  li:  HAS  BEEN  APPROVED 


ISSUE  6: 

Specifications  currently  permit  no 
latitude  or  judgment  by  customer  demands 
for  the  use  of  the  word  "shall"  in  place 
of  "will",  "should"  and  "may".  All 
stated  requirements  have  equal  weight  and 
each  must  be  fully  met.  However,  the 
real  world  is  that  some  requirements 
directly  conflict  with  each  other  or  are 
desires  which,  if  not  fulfilled,  have 
minimum  impact  upon  mission  achievement. 
Isn't  there  a  case  for  including  weighing 
factors  within  the  specification  such 
that  attention  is  directed  to  the  primary 
mission  success  factors?  Paragraph 
4.3.8  of  MIL-STD-490  permit  such  goal 
setting  for  Development  Specifications. 


ISSUE  6 


SHOULD  WEIGHING  FACTORS  BE  INCLUDED  IN 


SPECIFICATIONS  TO  GUIDE  THE  DESIGNER'S  EFFORT’ 


ISSUE  7: 

We  have  seen  a  trend  towards  the  require¬ 
ment  for  more  specifications  at  various 
levels  in  order  to  "better  manage"  the 
products,  i.e.,  control  the  configuration. 
This  has  several  side  effects:  the  more 
specifications  the  more  redundancy  in 
stated  requirements.  The  more  con¬ 
figuration  items  selected,  the  more  ECPs 
to  be  written  per  DOD-STD-480.  Effort 
is  diverted  from  mission  assurance  to  one 
of  ensuring  compatibility  across  documents. 
Why  not  limit  specification  preparation  to 
those  the  contractor  would  do  if  there 
no  CDRL  requirement? 

ISSUE  7 

SHOULD  THE  QUANTITY  AND  LEVELS  OF  SPECIFICATIONS  BE 
LIMITED  TO  THOSE  THE  CONTRACTOR  WOULD  WRITE  FOR 
HIS  OWN  USE’ 

ISSUE  8 

SHOULD  THERE  BE  A  COMPOSITE  HARDWARE  SOFTWARE 
SPECIFICATION  TYPE  BELOW  THE  SYSTEM  SEGMENT  LEVEL’ 

IF  SO  SHOULD  IT  ALSO  BE  USED  FOR  FIRMWARE’ 


ISSUE  8: 

The  only  specification  types  that  recognize 
both  hardware  and  software  components  are 
the  system/ system-segment  specifications. 
Below  that  level  the  specification  types 
are  either  hardware  or  software.  With 
today's  rapid  spread  of  programmed  and 
specialized  hardware/software  components, 
should  there  not  be  a  specification  type 
that  recognizes  hardware/software  com¬ 
binations  below  the  system  level?  The 
combined  specification  could  also  be  used 
for  firmware  devices. 

This  concludes  the  presentation.  Let  us 
now  go  back  and  address  each  issue  giving 
you  the  chance  to  state  your  opinion. 

Mr.  Wade  and  I  will  act  as  monitors  and 
watch  the  time  to  ensure  some  time 
remaining  for  addressing  any  issue  you 
desire  to  present. 


DESIGNING  FOR  MISSION  ASSURANCE 


SUMMARY  OF 

ISSUES  AND  RECOMMENDATIONS 


1.  Design  Reviews 
Issue 

Industry  perceives  that  the  effect¬ 
iveness  of  the  design  reviews  is 
hindered  by  lack  of  preparedness 
and  discipline  on  the  part  of  the 
Government  review  team.  The  use 
of  design  reviews  for  tutorial 
sessions  to  educate  the  customer 
can  be  counter-productive  to  the 
primary  objective  of  the  design 
reviews. 

Recommendations 

o  The  appropriate  Government  pro¬ 
gram  manager  should  insure  that 
a  small,  well  qualified  review 
team  is  formed  and  briefed  as  to 
their  responsibilities  prior  to  the 
start  of  the  review.  Insure  that 
the  team  members  are  familiar 
with  the  contractural  require¬ 
ments  of  MIL-STD- 1 52 1 A  or 
NASA  equivalent.  Insure  that 
team  members  have  reviewed  the 
documentation  (design  analysis 
reports'  prior  to  the  design  re¬ 
views.  Insure  that  the  Govern¬ 
ment  technical  experts  are  in¬ 
volved  early  in  the  contractor 
review  process  in  more  informal 
technical  sessions.  Consider  de¬ 
veloping  a  document  to  standard¬ 
ize  the  responsibilities  of  a  re¬ 
view  team  for  SDR's,  PDR's, 
CDR's,  etc. 

2.  Design  Reviews 
Issue 

The  Government  perceives  that  the 
design  reviews  have  degenerated 
to  a  public  relations  show  to  relate 
how  "good"  the  contractor  has  per¬ 
formed  rather  than  a  presentation 
of  the  facts  so  that  the  review  team 
(contractor  and  customer'  can  un¬ 
cover  design  errors.  There  is  no 


attempt  to  cover-up  design  errors 
but  the  emphasis  seems  to  be  on 
presenting  the  non-controversial 
rather  than  the  areas  of  relative 
unce  rtainty. 

Recommendations 

o  The  contractor  program  manager 
should  review  the  purpose  of  the 
design  reviews  and  resist  the 
P.  R.  tendency.  He  should 
realize  that  the  uncovering  of 
design  errors  is  not  necessarily 
a  reflection  on  contractor  com¬ 
petence.  The  Government  should 
of  course  also  recognize  this. 

o  Format  of  design  review  sessions 
should  include  adequate  time  for 
splinter  sessions  where  a  more 
detailed  review  by  a  smaller 
group  of  technical  experts  in  a 
specific  discipline  can  meet  and 
discuss  problems  or  areas  of 
concern  on  a  one-on-one  basis. 

3.  Design  Reviews 
Issue 

Over  the  years  the  design  reviews 
have  developed  into  an  advesary 
relationship  between  the  contractor 
and  the  Government.  This  polar¬ 
ization  can  be  detrimental  to  de¬ 
veloping  a  truly  effective  approach 
to  Mission  Assurance.  NASA 
statistics  over  20  years  show  65% 
of  all  avoidable  errors  are  design 
errors. 

Recommendations 

o  Build  the  team  approach.  Recog¬ 
nize  that  human  beings  make 
mistakes.  The  purpose  of  design 
and  readiness  reviews  is  to  catch 
and  uncover  any  errors.  This 
can  best  be  accomplished  in  a 
cooperative  manner  and  by  avoid¬ 
ing  finger  pointing  and  retribution. 
This  change  of  attitude  and  ap¬ 
proach  requires  implementation 
and  support  at  the  highest  levels 
of  the  contractor  and  the  Govern¬ 
ment. 


Design  Reviews 
Issue 

MIL-STD- 1 52 lA  has  become  too 
general  to  be  of  real  use  in  the 
space  business. 

Recommendations 

o  Consider  developing  a  Space 
Division  Standard  which  combines 
1521A  and  the  readiness  reviews 
required  by  SAMSO  Reg.  550-15. 
This  would  combine  and  put  in  one 
reference  document  all  the  tech¬ 
nical  reviews  of  a  particular  pro¬ 
gram.  It  would  insure  reviews 
which  are  consistent  with  space 
type  products.  It  would  insure  a 
continuity  of  the  program  from 
the  early  system  requirements 
review  to  the  mission  readiness 
review. 

Design  Reviews 
Issue 

Government  should  know  who  the 
responsible  individuals  in  various 
technical  disciplines  are  and  have 
access  to  them  without  excessive 
"filtering"  by  program  offices. 

Recommendations 

o  Contractors  should  identify 
"principal"  engineer  or  "cogni¬ 
zant"  engineers  in  each  technical 
discipline.  Direct  Government 
contact  with  these  individuals 
should  be  encouraged  but  not 
overexploited  so  that  it  prevents 
the  contractor  from  performing 
his  tasks. 

o  Early  identification  of  problems 
and  a  helpful  and/or  assisting 
attitude  by  the  Government  is  de¬ 
sired  rather  than  a  policing 
action.  This  modus  operandi  en¬ 
courages  further  teamwork, 
understanding,  and  problem 
resolution. 


6.  Design  Reviews 
Issue 

Design  changes  made  after  a  major 
review  do  not  always  get  a  thorough 
review.  These  "innocent"  changes 
are  sometimes  the  cause  of  Mis¬ 
sion  failure  since  the  subtleties  of 
the  change  and  its  effect  on  other 
systems  may  not  be  recognized  by 
the  originating  function. 

Recommendations 

oAll  changes,  both  Class  I  &  II, 
should  be  re- reviewed  prior  to 
DD-250.  This  can  be  done  in  a 
"systems  review"  or  as  a  part 
of  the  DD-250  selloff  process. 

All  software  changes,  no  matter 
how  minor,  should  be  reviewed 
by  the  contractor  in  an  engineer¬ 
ing  review  board  and  again  by  a 
Government/Contractor  working 
group  on  a  periodic  basis. 

7.  Design  Reviews 
Issues 

The  use  of  the  award  fee  must  be 
carefully  thought  out  so  that  design 
reviews  are  not  motivated  to  be  PR 
shows.  If  award  fees  are  based  on 
how  well  the  contractor  fared  at  a 
design  review,  the  contractor  will 
naturally  promote  the  good  and 
supress  the  bad  areas.  This  is 
counter-productive  to  the  purpose 
of  a  design  review. 

Recommendations 

o  Separate  the  award  fee  evaluation 
process  from  the  design  reviews. 

8.  Design- Designing  for  the  Shuttle 
ERA 

Issue 

There  is  a  lack  of  timeliness  and 
visibility  of  derived  design  re¬ 
quirements,  including  changes. 


for  payloads  using  the  space 

transportation  system. 

o  The  development  of  a  suitable 
definition  of  the  structural  loads 
environment  is  a  complex  task 
involving  many  assumptions.  At 
the  present  time  insufficient  in¬ 
sight  exists  into  shuttle  loads 
methodology  to  justify  confidence 
on  the  users  part  that  the  load  de¬ 
finition  for  payloads  is  adequate 
to  assure  mission  success. 

o  The  impact  of  an  unanticipated 
error  in  the  loads  environment 
definition  or  an  unanticipated  in¬ 
crease  in  predicted  loads  late  in 
the  program  as  a  result  of  model/ 
methodology  revisions  is  enor¬ 
mous  for  user  programs  i.  e.  , 
cost  increases  and  schedule  slips 
in  the  best  case.  On  the  other 
hand,  overly  conservative  ap¬ 
proaches  can  severly  impact 
mission  objectives. 

o  Current  methodology,  as  well  as 
contractural/procedural  short¬ 
comings,  leads  to  significant 
delays  in  meeting  identified 
spacecraft  needs,  particularly 
for  additional  requested  optional 
services.  A  specific  example  is 
shuttle  dynamic  model /forcing 
function  updates  to  support  pay- 
load  load  cycle  calculations  prior 
to  the  final  preflight  verification 
cycle. 

o  Because  of  the  complexity  of  the 
problem,  flight  verification  of  the 
loads  methodology  is  necessary 
for  mission  assurance  on  future 
flight  articles.  This  requires  an 
integrated  program  of  data 
acquisition,  data  analysis,  flight 
simulation  and  data  evaluation. 
Limited  instrumentation  is 
currently  planned  for  evaluation 
of  the  payload  environment,  and 
details  of  the  required  integrated 
effort  have  not  yet  been  complete¬ 
ly  defined. 


Recommendations 

o  Establish  an  Ad-Hoc  review  panel 
for  structural  loads  chaired  by 
AFSD/NASA  with  membership  to 
include  payload  contractors. 

o  Review  the  need  for  similar 
panels  for  other  induced  environ¬ 
ments. 

o  The  intent  in  forming  these  Ad- 
Hoc  review  panels  is  to  bring 
together  involved  organizations 
for  an  in  depth  group  evaluation, 
reducing  the  need  for  NASA  to 
respond  to  individual  program 
requests. 

Suggested  Review  Panel  Tasks 
Include: 

o  Perform  a  comprehensive  review 
of  the  present  basis  for  establish¬ 
ing  the  structural  loads  environ¬ 
ment  for  shuttle  payloads.  De¬ 
velop  recommendations  /  task 
statements  to  resolve  any  prob¬ 
lem  areas,  oversee  the  accom¬ 
plishment  of  these  tasks,  and 
evaluate  the  results. 

o  Perform  a  comprehensive  review 
of  payload  flight  environment 
verification  plans  including  data 
acquisition,  data  analysis,  flight 
simulation,  and  data  evaluation. 
Develop  recommendations  /  task 
statements  to  resolve  any  per¬ 
ceived  deficiencies,  oversee  the 
accomplishment  of  any  defined 
tasks,  and  evaluate  any  results. 

o  Develop  recommendations  /  task 
statements  on  approaches  for  re¬ 
ducing  costs  and  improving  the 
timeliness  of  such  optional 
services  as  providing  updated 
shuttle  models /forcing  functions 
to  support  program  loads 
analyses.  Oversee  accomplish¬ 
ment  of  these  tasks  and  evaluate 
any  results. 


o  Provide  appropriate  recommenda¬ 
tions  on  methodology  revisions, 
if  any. 

o  Provide  recommendations  re¬ 
garding  the  needs  for  maintaining 
a  periodic  review  of  these  areas. 

oAdequate  funding  for  the  tasks  re¬ 
commended  by  the  panels  needs 
to  be  provided. 

Design-Failure  Analysis 

Issue 

Effective  failure  analysis  is  a 

valuable  tool  in  making  critical 

management  decisions. 

Recommendations 

oThe  program  manager  must  be 
appraised  of  potential  problems 
and  the  work  going  on  in  the 
failure  analysis  of  products  in 
his  program  so  that  he  has  a 
good  understanding  of  the  tech¬ 
nical  aspects  of  the  problem  and 
options  open  for  solution  and 
correction. 

o  The  request  for  proposal  (RFP) 
should  define  the  requirements 
for  failure  analysis,  and  failure 
modes  and  effects  analysis 
(FMEA). 

o  Contractors  must  be  explicit  in 
what  he  will  do  on  failure  analysis 
work  in  res  ponse  to  what  the 
customer  has  defined  in  his  RFP 
or  SOW.  Final  negotiations  must 
iron-out  the  customer-contractor 
agreed-to  work  and  tasks  in 
failure  analysis. 

oAdequate  funding  must  be  pro¬ 
vided  for  failure  analysis  efforts; 
program  office  must  be  fair  in 
their  distribution  of  such  funds  or 
funds  should  come  from  a  func¬ 
tional  organization  source. 


10.  Design  -  Failure  Analysis 
Issues 

Expanded  industry  exchange  of  in¬ 
formation  on  significant  failure 
analyses  and  problems  would  save 
time  and  money  and  improve 
failure  analysis  effectiveness. 

Recommendations 

o  Conduct  study  to  determine 
practicality  of  developing  an 
industry  reference  source  to 
foster  such  information  exchange 
between  contractors,  Space 
Division,  and  NASA  space  activ¬ 
ities  beyond  GIDEP  and  program- 
only  data  banks. 

11.  Design  -  Failure  Analysis 
Issue 

Who  should  have  the  primary  re¬ 
sponsibility  for  conducting  failure 
analysis  ?  The  basis  issue  is 
whether  or  not  the  program  office 
should  have  primary  control. 

Recommendations 

o  Failure  analysis  should  be  done 
by  an  independent  group  (unbiased' 
and  not  in  Program  Office,  but 
use  team  members  from  affected 
organizations.  However,  failure 
analysis  must  be  timely  and  com¬ 
municated  to  appropriate  organ¬ 
izations.  Also,  if  the  failure 
analysis  concerns  a  piece  part, 
the  manufacturer  or  supplier 
should  be  involved. 

oFor  complex  failure  analysis  pro¬ 
blems  a  team  approach,  includ¬ 
ing  customer  supplier  and  inde¬ 
pendent  agency  is  desireable. 

12.  Design  -  Failure  Analysis 
Issue 


There  is  a  need  for  procedural 
guidance  in  conducting  failure 


analysis  to  identify  various 
successful  methods  and  techniques. 

Recommendations 

o  The  Air  Force  Rome  Air  Develop¬ 
ment  Center  (RADC)  is  developing 
a  guidance  document  entitled 
■'Failure  Analysis  Techniques''. 
This  document  is  scheduled  for 
release  during  1980.  Ordering 
and  use  of  this  document  is  re¬ 
commended. 

Design/ Configuration  Item 
Specifications 

Issue 

Should  MIL-STD-483,  490  Part  II 
(product)  specifications  be  required 
when  there  is  no  need  for  down¬ 
stream  production  or  second 
sourcing  ? 

Recommendations 

o  Provide  tailoring  guidance  for 
eliminating  the  need  to  include 
requirements  for  a  Part  II 
specification  in  the  Contract  Data 
Requirements  List  (CDRL)  and 
discourage  its  use  for  Space 
Division  contracts  whenever 
appropriate.  Suggested  method  is 
an  AFSC  Policy  and  pamplet  on 
use  of  specifications  for  manag¬ 
ing  a  program  and  suggestions 
for  tailoring  to  fit  various  kinds 
of  programs  should  be  issued  to 
guide  program  application. 

Design/Configuration  Item 
Specifications 

Issue 

Software  (computer  program)  Spec¬ 
ifications  are  not  used  for  pur¬ 
poses  identical  to  hardware.  They 
require  that  "As-Built  ”  data  be 
included  which  results  in  a  require¬ 
ments  and  compliance  combination 
document 


Recommendation 

oAFSD  should  sponsor  a  review 
of  software  specification  require¬ 
ments  to  make  compatible  with 
hardware  specifj<~ation  require¬ 
ments. 

15.  Design/ Configuration  Item 
Specifications 

Issue 

MIL-STD-483  and  490  do  not  re¬ 
cognize  inclusion  of  development 
and  product  specification  materi¬ 
al  in  the  same  document  except 
as  a  Part  I/Part  II  combination 
with  each  part  a  complete  six 
section  specification.  'Why  not 
allow  a  combination  specification 
with  functional  and  fabrication 
requirements  ? 

Recommendations 

o  Permit  updates  to  authenticated 
system  and  development  specifi¬ 
cations  (Class  I  and  Class  II,  Ref. 
Issue  No.  4'  to  document  develop¬ 
ment  including  completion  of  to- 
be-determined  (TBD'  items. 

o  Permit  combination  functional'' 
fabrication  specifications  to  in¬ 
corporate  usage  of  existing  items 
or  modification  of  existing  items, 
into  a  new  system  application. 

o Issue  tailoring  guidance  instruc¬ 
tions  to  incorporate  above  re¬ 
commendations. 

16.  Design/Configuration  Item 
Specifications 

Issue 

DOD-STD-480  permits  no  change 
to  authenticated  baseline  specifi¬ 
cations  unless  approved  as  a  Class 
I  change.  (Ref.  4.  2.  I.  2  a  and  b  of 
DOD-STD-480  and  the  definitions 
in  110-2,  110-30  and  110-43'. 
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Recommendations 


o  Issue  tailoring  instructions  to 
permit  tailoring  DOD-STD-480A 
to  authorize  use  of  a  Class  II 
change  to  incorporate  minor  re¬ 
visions.  Such  Class  II  changes 
to  be  approved  by  the  customer's 
plant  representative  and  furnish¬ 
ed  to  the  customer  as  an  update 
to  the  CDRL  delivered  data  item 
(the  specification's  requirement'. 

7.  Design/ Configuration  Item 
Specifications 

Issue 

All  specifications  must  be  main¬ 
tained  throughout  the  program's 
life  cycle  even  though  effectively 
replaced  by  lower  tier  approved 
documents. 

Recommendations 

o  Provide  tailoring  guidance  for 
MIL-STD-483  which  will  define 
permissable  elimination  of  spec¬ 
ification  maintenance  require¬ 
ments  when  lower  level  specifi¬ 
cations  have  effectively  replaced 
them, 

8.  Design/ Configuration  Item 
Specifications 

Issue 


19.  Design/Configuration  Item 
Specifications 

Issues 

Why  require  more  specifications 
than  the  contractor  requires  for 
his  own  internal  use  ? 

Recommendations 

o  Require  justification  for  any  re¬ 
quirement  for  specifications  that 
the  contractor  would  not  write 
if  not  included  within  the  CDRL 
as  a  required  specification.  Pro¬ 
vide  guidance  strongly  encourag¬ 
ing  use  of  specifications  normal 
to  the  contractor's  system. 

20.  Design  /  Configuration  Item 
Specifications 

Issue 

MIL-STD's  have  no  provision  for  a 
composite  hardware /software  item 
below  the  system/system- segment 
level. 

Recommendations 

o  Revise  MIL-STD-480  and  483  to 
define  development  and  product 
specifications  or  combinations 
thereof  for  items  which  contain 
both  hardware  and  software  com¬ 
ponents. 


Order  of  requirements  precedence 
and  requirements  other  than  "shall" 
are  seldom  used  even  though  per¬ 
mitted  by  the  MIL-STD's. 

Recommendations 


o  Promote  usage  of  the  order  of 
requirements  precedence  section 
of  the  specification  and  use  words 
such  as  "should"  and  "may"  or 
establish  goals  as  goals. 

o  Include  in  AFSD's  specification 
preparation  manuals. 
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Principal  Policy  References 

Requirement  for  Subcontract  Management 
(AFSC  DAR  Sup  23-5000) 

Requirement  for  Subcontract  Management 
Plan  -  RF^s 

(AFSC  DAR  Sup  7-2003.100) 

Subcontract  Management  Clause  - 
Contracts 

(AFSC  DAR  Sup  7-150.4) 

Supplier  Quality  Assurance  Program 
Requirements 

(MIL-STD-1535A) 


Award  Fee 

SD  Pamphlet  70-9,  Award  Fee  Guide 
-CPAF,  CPIF-AF,  FPIF-AF,  FFP-AF 
-Systems  and  Subcontract  Management 
-Control  of  Subcontractor  Design, 
Performance,  and  Schedule 


Memorandum  of  Agreement/Letter  of 
De legation 

Requires  AFPRO/DCAS  to  flow  down 
surveillance/ reporting  requirements 
for  visibility  purposes. 


Reviews  at 

Subcontractor  Plants 

MM/PCR 

PDR/CDR 

PRR 

Problem  Solving 

PAS 

Management  Meetings 

SUBCONTRACT  MANAGEMENT  PROBLEMS 
A  PROGRAM  OFFICE  PERSPECTIVE 
by  Col  Stan  Kennedy 
Director  of  Engineering 
Defense  Support  Program 
Air  Force  Space  Division 


Mission  Assurance  -  1978 
DSP  Attendees: 


Engineering  7 
Config  Mgmt  2 
Prog  Control  1 
Contracting  2 
Aerospace  3 


DSP  Contractors: 
President/Gnl  Mgr  2 
Quality  Assurance  3 
Material  1 

Engineering  1 

Subcontract  Mgr  1 


Mission  Assurance 


Objective:  Achieve  specification 

performance  on-orbit  up 
to  and  beyond  design  life. 

Program  Office: 

o  Evolves  a  highly  redundant  design, 
o  Eliminates  all  single  point 
failures . 

o  Develops  a  comprehensive  test 
program. 

o  Conducts  independent  readiness 
review . 

Problem  Areas 

Prime:  Over  Spec 

Inadequate  Monitoring 
Lack  of  Incentives 

Sub:  Lack  of  Motivation 

Ineffective  Management 
Misuse  of  Priority 
Marginal  Performance 


Management  Relationship 
Prime-To-Sub 

Prime: 

Requires  Visibility 
o  Engineering  -  Spec/Performance 
o  Schedule  -  Milestones 
Priorities 


Sub: 

Requires  Visibility 
o  Performance 
o  Motivation 
o  Need  Date 


Solutions  (?) 

Communications:  Tailored  Specs 

Firm  Milestones 
Monitor  Progress 

Incentives:  Near  Term 

Motivators 

Quality  Controls:  Tailored  Standards 
AFPRO/DCAS 

Organization:  Responsive  to  Customer 


r 


PRESIDENT 


ir 


7 


ENGINEER 


PROGRAM 


PRODUCTION 


MATERIAL 


PL95-507  IMPLEMENTATION 
CONTRACTING  VIEWPOINT 
by  Col  Thomas  Van  Meter 
Ballistic  Missile  Office 

I.  Three  areas  for  discussion: 

A.  Source  of  requirements. 

B.  Key  points  that  need  more 
emphasis  in  preparing  small  and  dis¬ 
advantaged  business  subcontracting 
plans  for  each  contract. 

C.  Role  of  the  mission  assurance 
community . 

II.  Requirements  are  listed  in  consider¬ 
able  detail  in  the  statute  itself.  Thus 
compliance  is  a  matter  of  law  and  not 
just  a  regulatory  requirement.  The  PCO 
must  assure  that  the  plan  is  submitted 

on  contracts  and  modifications  valued  at 
more  than  $1  million  on  construction  of 
a  public  facility  or  $500  thousand  on 
others. 

A.  Approve  the  adequacy  of  the  plan. 
(Seeks  advice  from  the  administrative 
contracting  officer,  the  Small  Business 
Administration  local  representative,  and 
the  DoD  Small  and  Disadvantaged  Business 
spec iallst .) 

B.  Review  prior  performance  in  meet¬ 
ing  small  and  disadvantaged  business 
goals  as  a  part  of  the  responsibility 
determination. 

C.  Determine  the  Inductive  fee  where 
appropriate. 

III.  Key  elements  that  need  to  be 
emphasized  in  preparing  the  plan: 

A.  Set  forth  goals  for  small 
business  as  a  percentage  of  the  total 
subcontracting  effort.  Include  sub¬ 
contracts  charged  directly  to  the  con¬ 
tract  and  a  proportionate  share  of  the 
indirect  subcontracts. 

B.  Name  the  program  administrator 
and  describe  duties  and  responsibilities. 
Describe  access  of  the  administrator  to 
the  top  level  executives. 

C.  Describe  contractor's  program 
and  what  is  being  done  to  seek  out  and 
develop  small  and  disadvantaged  business 
sources.  The  plan  needs  to  list  the 
prospective  or,  when  known,  the  actual 
sources,  type  of  work  performed,  dollar 
values,  and  periods  of  performance. 


D.  Be  prepared  to  report  results 
and  degree  of  success  in  meeting  goals. 

E.  Describe  records  kept  so  that 
results  can  be  reviewed. 

IV.  Product  assurance  community  role: 

Help  administrators  and  procurring  con¬ 
tracting  officers  establish  attainable 
goals,  but  yet  ones  which  require  empha¬ 
sis  and  management  attention  to  attain. 

Remember  three  ”Cs”: 

-  Candidates — find  potential  sources  in 
both  the  technical  and  non-technical 
areas. 

-  Credibility — establish  attainable  goals 
Help  firms  out.  Tailor  to  needs  of  the 
firms,  and  don't  oversell  or  undersell 
ability  to  meet  goals. 

-  Competence — help  small  and  disadvan¬ 
taged  firms  to  succeed. 


SUBCONTRACT  MANAGEMENT 
Donald  R.  Fowler 
Major,  USAF 

AF  Contract  Management  Division 

•  AF  CONTRACT  MANAGEMENT  DIVISION  (AFCMD) 
Kirtland  AFB,  New  Mexico 

o  One  of  three  Contract  Administration 
Servi ces 

--Defense  Contract  Administration 
Service  (DCAS) 

--Navy  Plant  Representative 
Offices 

--Air  Force  Plant  Represen ta t ive 
Offices 

o  Composed  of  20  AFPROs 
o  Missions 

--Program  Support 

--Management  systems  surveillance 
o  AFPRO  Functional  Divisions 
--Quality  Assurance 
--Engineering 

--Manufacturing  Operations 
--Government  Property 
- -Contracts 

--Subcontracts  Management 

•  SUBCONTRACT  MANAGEMENT  DIVISION  (SM) 
o  Mission 

--Program  Support 

--Procurement  system  surveillance 
o  Origin 

--Need  during  1960s 
--USAF  generals'  initial  concept 
--1973  formed  from  other  functions 
o  Initial  SM  Purposes 

--Baseline  procurement  system 
--Purely  after-the-fact 

•  SM  TODAY  (STILL  MULTI-FUNCTIONAL) 

o  SM  accepted  as  a  true,  necessary 
function 


o  SPO  support  is  priority 
o  Haior  Problems  in  SM  today 
- -Manpower / 1  urnover 

--Past  -  no  "need"  for  SM!  (C-5A  sub) 
--Only  AFPROs  have  SM~dedicated 
--No  training  courses  for  SM 

•  PRE-AWARD  PHASE  OF  THE  SUBCONTRACT 
o  Pre-award  surveys 

o  Past  subcontractor  performance 

o  Type  of  subcontract  based  on  risk 
and  prime  (associate  contractors) 

o  RFP  review  and  subcontract  management 
plan 

o  Emphasis  on  smal 1 / disadvantaged 
business 

o  Assess  statements  of  work 

•  AWARD  PHASE  OF  THE  SUBCONTRACT 

o  Purchase  Order,  Interdivis ional 
Work  Authorization  and  Consent-to- 
Issue  Reviews 

--Purchase  Order  and  subcontract 
defined 

--Assess  "flowdowns"  (some  needless 
costs) 

--Designate  major/critical  sub¬ 
contracts  and  Interdivis ional 
Work  Authorizations 

--Assess  cost/price  analysis  and 
negotiat ions 

--Assess  all  functional  flowdowns 

o  Initiate  multi-functional  delegations 

t  POST-AWARD  PHASE  OF  THE  SUBCONTRACT 

o  C/Sese  (what  SPO  needs  most) 

o  Reports  from  delegations 

o  Closer  coordination  with  SPOs 

o  Assess  prime  contractor  managment 
of  subs 

o  Visits  to  subcontractors 

o  Aid  in  manufacturing  management 
production  capability  review.s 

o  Aid  in  production  readiness  reviews 

o  Strikes  and  workarounds 


o  QA  back  to  QA 


o  Shoriages  and  long- lead  planning 


•  BlC.-aiFITS  TO  CONTRACTORS 

o  Intermediary  between  SPO  and 
prime 

o  Hopefully  "one  face  to  industry" 

o  ASPS  No.  1  being  rewritten  (less 
s  ub j  ectivity) 

•  CONTRACTOR  REACTION  TO  SM 

o  Some  ignore  consents  ("enough 
bus  iness" ) 

o  Nobody  likes  auditors 

o  Imposed  corrective  action 

•  FUTURE  OF  SUBCONTRACT  MANAGEMENT 

o  50%  of  prime  dollars  in 
subcontracts 

o  Prime's  overhead  forces  subs 
work 


NOTE:  SPO  support  can  be  defined  as: 
teamwork  in  the  acquisition  process  for 
wisely  spending  Government  money;  a 
process  of  saving  the  SPO's  time; 
an  avoidance  of  duplicate  effort  between 
SPO  and  AFPRO;  a  common  sense  business 
approach  based  on  coordination,  communi¬ 
cation.  and  cooperation;  caking  the 
lead  in  the  phases  of  acquisition  best 
suited  for  one's  discipline;  a  cross- 
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1.  INTRODUCTION 

In  order  to  adequately  appreciate  the 
Centaur  Program's  mission  assurance  sys¬ 
tem,  it  is  essential  to  first  present  a 
summary  of  the  program's  history,  respon¬ 
sibility  and  structure.  The  Centaur  has 
been  Integrated  with  the  Atlas  and  Titan 
IIIE  boosters  in  what  will  be  referred  to 
herein  as  the  Centaur  Program.  The  Cen¬ 
taur  itself  is  an  upper  stage  launch  vehi¬ 
cle  having  a  thrust  of  about  30,000  lbs. 
provided  by  two  engines  burning  liquid 
oxygen  (L02)  and  liquid  hydrogen  (LH2) . 

Its  development  began  in  the  fifties  when 
the  Lewis  Research  Center  (LeRC)  did  much 
pioneering  work  in  developing  liquid  hydro¬ 
gen  technology.  In  1953,  an  experimental 
L02/LH2  engine  with  5,000  lbs.  thrust  was 
fired  at  LeRC.  General  Dynamics  Convair 
Division  (GDC)  was  awarded  contracts  to 
design  the  Centaur  launch  vehicle,  and  in 
1963  the  first  successful  flight  of  Cen¬ 
taur  D  atop  an  Atlas  booster  occurred 
under  technical  management  of  LeRC.  Cen¬ 
taur's  first  mission  to  inject  Surveyor 
moon  landers  into  their  translunar  orbit 
was  an  outstanding  success. 

During  the  early  seventies,  a  D-1  im¬ 
proved  Centaur  Program  was  completed.  The 
objectives  of  the  D-1  were  to  achieve 
lower  boost  vehicle  integration  costs. 

Ilic  Centaur  D-IA  (Centaur  D-1  atop  an 
\tlas  SLV-3D)  achieved  operational  status 
with  its  first  launch  of  Pioneer  II  (AC- 
30)  in  April  of  1973.  The  Centaur  D-IT 
(Centaur  D-1  atop  a  Titan  lllE)  became 


operational  with  the  Helios  A  launch  in 
December  of  1974.  The  Centaur  D-IA  and 
D-IT  employ,  in  general,  nonredundant 
systems.  This  combined  with  the  impor¬ 
tance  of  its  payloads  made  it  imperative 
that  the  program  adopt  and  maintain  a 
stringent  program  mission  assurance  sys¬ 
tem. 

The  Centaur  D,  D-IA  and  D-IT  have 
been  used  to  transport  into  space  a  vari¬ 
ety  of  payloads.  Since  the  operational 
phase  for  Centaur  began  in  1968,  there 
have  been  50  launches  using  the  Atlas/ 
Centaur  and  Titan/Centaur  of  which  there 
have  been  5  failures.  Only  two  of  these 
five  failures  were  caused  by  a  malfunction 
of  the  Centaur  upper  stage.  The  Centaur 
has  been  used  for  missions  sponsored  by 
NASA,  DOD,  foreign  countries,  and  private 
corporations.  The  Titan/Centaur  has  been 
responsible  for  successfully  placing 
several  prestigious  spacecraft  into  a 
proper  trajectory  to  complete  its  mission, 
such  as  Helios  A  and  B  (probes  to  the 
Sun  sponsored  by  the  West  German  govern¬ 
ment),  Viking  A  and  B  (spacecraft  that 
soft  landed  on  Mars)  and  Voyager  A  and  B 
(space  probes  to  the  large  outer  planets 
of  our  solar  system) .  These  missions  not 
only  represent  an  investment  of  more  tlian 
a  billion  dollars,  but  also,  in  some  cases, 
had  to  be  launched  without  significant 
compromises  to  mission  opportunities/ 
launch  windows.  The  Atlas/Centaur  has 
been  responsible  for  successfully  placing 
Into  geosynchronous  orbit  several  com¬ 
munication  satellites  for  DOD,  INTELSAT 
and  COMSAT.  To  a  very  great  extent,  the 
existence  of  today's  international  and 
domestic,  communications  network  is  the 
result  of  a  successful  Atlas/Centaur 
launch  vehicle  program. 

In  general,  the  Centaur  Program's 
structtire  is  not  unusual  as  far  as  gov¬ 
ernment  programs  are  concerned.  NASA's 
Office  of  Space  Transportation  Systems 
has  assigned  a  division  of  NASA's  Lewis 
Research  Center  with  the  responsibility 
of  (1)  integrating  the  launch  vehicle 
system  with  the  payload,  (2)  procuring 
the  launch  vehicle  hardware,  (3)  assuring 
the  overall  quality  of  the  vehicle,  (4) 
managing  all  prelaunch  and  launch  activi¬ 
ties  and  (5)  managing  all  problem  solving 
activities  and  necessary  hardware  develop¬ 
ment.  General  Dynamics  is  LeRC's  prime 
contractor  and  is  responsible  for 
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fabricating  the  Centaur  vehicle  and  Atlas 
booster.  LeRC  also  uses  the  following  as¬ 
sociate  contractors  to  supply  GDC  with 
the  Government  Furnished  Equipment  (GFE) 
listed  below: 

1.  Rocketdyne  Division,  Rockwell 
International  -  Atlas  Engines 

2.  Pratt  &  Whitney  Aircraft  - 
Centaur  Engines 

3.  Teledyne  Systems  Company  - 
Digital  Computer  Unit  and  Remote  Multi¬ 
plexer  Units 

4.  Honeywell,  Inc.  -  Inertial  Plat¬ 
form  and  Electronics 

Needless  to  say,  all  of  the  above  contrac¬ 
tors  use  the  services  of  numerous  subcon¬ 
tractors  and  vendors.  Also,  the  Centaur 
Program  uses  to  as  great  an  extent  as 
possible  delegated  government  agencies 
such  as  the  Defense  Contractor's  Adminis¬ 
trative  Services  (DCAS)  to  Inspect,  assure 
the  quality  of  hardware,  and  assure  com¬ 
pliance  to  requirements  of  hardware  speci¬ 
fications  at  the  contractors,  subcontrac¬ 
tors  and  vendors.  Finally,  NASA  uses 
facilities  and  personnel  at  Kennedy  Space 
Center  (KSC)  to  alunch  these  vehicles. 

The  program  structure  for  a  Titan/Centaur 
launch  vehicle  was  very  similar  with  the 
exception  that  the  Martin  Marietta  Corpora¬ 
tion  serves  as  an  associate  contractor 
supplying  GDC  with  a  Titan  IIIE  booster  in 
place  of  the  Atlas  booster. 

Program  related  activities  of  the 
above  organizations  are  centered  around  a 
document  called  the  Mission  Assurance 
Plan  (MAP) .  The  MAP  is  a  contractor  pre¬ 
pared  and  government  approved  document 
that  describes  tasks  the  contractor  will 
do  to  assure  mission  success.  The  MAP  is 
based  on  the  government's  contract  re¬ 
quirements,  the  contractor's  operational 
methods  and  organizational  elements  in¬ 
volved.  The  MAP  is  divided  into  manage¬ 
ment,  engineering,  and  quality  tasks  and 
for  each  task  Indicates  customer  require¬ 
ments,  implementing  responsibilities,  im¬ 
plementation  methods,  evaluation  criteria, 
reference  to  applicable  specifications, 
and  reference  to  customer  documents  that 
give  detailed  instructions.  The  MAP  was 
prepared  by  GDC  for  NASA  and  the  Space 
Division  (SD)/USAF.  The  MAP  applies  to 


multiple  contracts  for  space  launqh  vehi¬ 
cles  and  related  services. 

Thus,  it  can  be  accurately  stated 
that  the  Centaur  Program  possesses  a 
structure  similar  to  most  large-scale 
high  reliability  programs.  Therefore,  as 
a  result  of  its  excellent  success  record 
(90  percent  success),  its  possessing  few 
redundant  systems,  and  its  complex  struc¬ 
ture,  the  mission  assurance  experiences 
of  the  Centaur  Program  should  be  of  in¬ 
terest  to  most  major  NASA  and  DOD  high 
reliability  programs. 

II.  CENTAUR  PROGRAM  MISSION  ASSURANCE 

SYSTEM _ 

The  success  associated  with  the 
Centaur  Program  is  the  result  of  the  pro¬ 
gram's  strict  adherence  to  sound  design 
practices  and  configuration  control, 
realistic  specifications,  its  establish¬ 
ing  the  team  concept  not  only  among  the 
government  and  contractor  personnel  but 
also,  where  possible,  at  the  subcontrac¬ 
tors  and  vendors,  and  its  ability  to  de¬ 
tect  and  resolve  problems.  It  would  be 
instructive  to  discuss  these  mission  as¬ 
surance  principles  as  they  apply  to  the 
Centaur  Program  in  more  detail.  Because 
the  Centaur  was  developed  by  NASA  for 
space  use,  and  the  Atlas  and  Titan  IIIE 
were  developed  by  DOD  for  weapons  use, 
these  principles  apply  in  different  de¬ 
grees  to  the  various  stages  of  the 
Centaur  Program  launch  vehicles.  For 
instance,  the  design  philosophies  out¬ 
lined  below  are  more  directly  related  to 
the  Centaur  rather  than  the  Atlas  or 
Titan  IIIE.  However,  the  Centaur  Program 
has  attempted  to  apply  these  principles 
to  all  the  stages  as  thoroughly  as  would 
be  reasonable.  Most  important  is  the 
fact  that  such  an  integration  of  launch 
vehicle  stages,  each  having  a  different 
heritage,  made  it  obvious  that  intense 
quality  verification  emphasis  would  be 
needed  for  mission  assurance. 

The  design  has  a  direct  effect  on  the 
reliability,  quality  and  safety  of  a  sys¬ 
tem,  In  its  design  phase,  close  atten¬ 
tion  was  given  to  the  Centaur  reliability, 
quality  and  safety  characteristics;  in 
many  cases,  by  engineers  specializing  in 
these  areas.  Many  different  analyses 
such  as  hazard  analyses,  worst-case 
analysis,  and  failure  modes  and  effects 


analyses  were  conducted  to  maximize  the 
Centaur's  reliability,  quality,  and 
safety  characteristics.  Also,  design 
criteria  which  specifies  safety/perfor¬ 
mance  margins  such  as  structural  factors 
of  safety  and  electronic  parts  derating 
were  established  before  the  design  was 
undertaken  and  strictly  adhered  to  during 
the  design.  The  design  of  all  program 
hardware  did  Include  consideration  of 
future  testing  in  order  to  provide  for 
obtaining  adequate  data  concerning  the 
test.  For  example,  test  points  were  con¬ 
veniently  located  in  hardware  so  that 
parameters  such  as  critical  voltages, 
currents,  pressures,  temperatures,  and 
flows  could  be  monitored  during  a  test 
with  the  smallest  possible  inconvenience. 

Testing  is  extremely  important  to 
and  occupies  a  large  part  of  the  Centaur 
Program  activities  to  ensure  reliability 
of  the  components,  subsystems,  systems 
and  the  overall  launch  vehicle.  The 
Centaur  Program  not  only  employed  exten¬ 
sive  testing  during  the  vehicle  hardware 
design  (Design  Verification  Testing)  and 
qualification  (Qualification  Tests)  but 
also  to  berify  the  quality  (Acceptance 
Testing)  of  all  hardware  and  each  vehicle. 
The  program  uses  design  verification  tes¬ 
ting  to  "fine  tune"  the  design  and  charac¬ 
terize  the  performance  of  the  design. 

Much  care  is  used  in  performing  flight 
qualification  tests  which  include  func¬ 
tional,  environmental,  stress  limit  and 
life  tests  to  insure  the  hardware  can 
satisfactorily  withstand  conditions  more 
severe  than  it  is  expected  to  experience 
in  acceptance  testing  and  normal  flight 
operations.  For  example,  in  vibration 
testing,  it  is  normally  required  that  the 
design  be  capable  of  surviving  at  least 
1  1/2  times  the  expected  flight  vibration 
levels.  Much  effort  was  expended  early 
in  the  program,  and  still  is,  to  seek  im¬ 
provements  in  acceptance  testing  to  in¬ 
sure  : 

1.  All  test  environments  exceed 
those  of  succeeding  tests  and  final  launch. 

2.  No  test  condition  degrades  the 
hardware , 

3.  All  hardware  is  tested  as  it  will 
perform. 

4.  All  hardware  is  adequately 


monitored  to  insure  proper  performance 
and  to  locate  any  malfunction. 

Programs,  no  matter  how  mature,  will 
always  experience  changes  in  design, 
materials  or  processes.  The  Centaur 
Program  is  no  exception  and  has  developed 
a  formal  method  for  controlling  the  con¬ 
figuration  of  the  vehicle.  The  Centaur 
configuration  system  can  be  broken  into 
the  three  main  functions, of  change  evalua¬ 
tion,  implementation  and  documentation. 

The  Centaur  Program  prime  and  associate 
contractors  are  required  to  submit  to 
LeRC  a  description  of  any  change,  justi¬ 
fication  for  change,  and  sufficient  data 
to  evaluate  the  effect  of  the  change  on 
the  vehicle  performance.  This  package  is 
evaluated  by  the  LeRC  engineer  responsi¬ 
ble  for  the  hardware  where  the  change  is 
required.  Once  he  is  satisfied,  he  sub¬ 
mits  the  change  requests  to  an  Engineer¬ 
ing  Review  Board  which  formally  rules  on 
whether  the  change  should  be  implemented. 
The  Engineering  Review  Board  is  a  formal 
board  composed  of  launch  vehicle  manage¬ 
ment  engineers  and  reliability  engineers. 
After  the  change  is  approved  by  the 
Engineering  Review  Board,  it  is  submitted 
to  the  Launch  Vehicles  Change  Board  (LVCB) 
where  it  is  evaluated  with  respect  to 
schedule  and  cost  as  well  as  technically. 
LVCB  also  assures  that  the  change  will  be 
properly  documented. 

Realistic  specifications  are  the  back¬ 
bone  of  the  Centaur  Program's  hardware. 
Specifications  must  on  one  hand  assure 
obtaining  the  most  reliable  piece  of 
hardware  and  on  the  other  hand  not  be  so 
restrictive  as  to  cause  the  most  quali¬ 
fied  vendors  to  not  bid  on  fabrication  of 
the  device.  Realistic  specifications 
are  developed  by  first  defining  the  criti¬ 
cal  materials,  processes,  and  testing  and 
then  controlling  these  items  adequately 
through  the  specification.  This  close 
scrutiny  of  the  specification  is  appli¬ 
cable  to  systems,  subsystems,  and  piece 
parts.  Finally,  ongoing  control  of  the 
materials,  processes  and  testing  must  be 
assured  by  the  contractor,  subcontractor 
or  government  representatives  (DCAS) . 

In  a  majority  of  cases,  the  Centaur 
Program  found  military  specifications 
satisfy  the  above  requrireraents .  There¬ 
fore,  they  should  be  used  as  frequently 
as  possible  in  a  high  reliability  program. 


mission  assurance 


In  certain  areas  (electronics  parts)  the 
Centaur  Program  found  military  specifica¬ 
tions  could  be  augmented  and  additional 
confidence  established  by  additional  hard¬ 
ware  testing.  Whenever  no  military  speci¬ 
fications  exist  to  control  an  item  needed, 
the  Centaur  Program  has  expended  consider¬ 
able  effort  to  develop  realistic  specifi¬ 
cations.  In  these  cases.  Centaur  Program 
contractor  and  government  representatives 
have  worked  with  subcontractors  and  ven¬ 
dors  to  develop  specifications  that  as:- 
sure  adequate  control  of  materials,  pro¬ 
cesses  and  testing.  These  negotiations 
are  usually  not  easy  and  require  careful 
compromises. 

One  of  the  most  important  assets  of 
the  Centaur  Program  is  the  team  concept. 
The  team  concept  on  the  program  has  been 
most  successfully  applied  to  the  group  of 
engineers  the  program  refers  to  as  system 
engineers.  Each  major  vehicle  system  has 
assigned  to  it  a  contractor  and  govern¬ 
ment  system  engineer.  With  respect  to 
that  system,  the  government  system  engi¬ 
neer  and  his  contractor  counterpart  are 
responsible  for  the  following: 

1.  Mission  Integration  Interfaces 

2.  Problem  Solving 

3.  Evaluation  and  Qualification 
Testing 

4.  Factory  Acceptance 

5.  Product  Reviews 

6.  Flight  Readiness  Status 

7.  Launch  Support 

8.  Flight  Data  Reduction  Analysis 

9.  Flight  Report 

10.  Contract  Support 

11.  Liaison  with  KSC  and  Contractor 

The  Centaur  Program  has  promoted  the 
team  concept  effectively  by  practicing 
the  following: 

1.  Showing  with  positive  action 
(recognition  awards,  promotions)  that  the 
system  engineer  is  important  in  achieving 


2.  Providing  definite  direction 
through  management  instructions  to  the 
systems  engineer. 

3.  Keeping  the  systems  engineer  in¬ 
formed  on  not  only  technical  issues  but 
management  Issues  also. 

The  Centaur  Program  has  found  that  devel¬ 
oping  the  team  concept  among  subcontrac¬ 
tors  and  vendors  much  more  difficult. 
Because  the  subcontractor  and  vendor  are 
more  removed  from  the  program  and  because 
their  main  job  and  interests  do  not  neces¬ 
sarily  align  themselves  with  those  of  the 
program,  they  can  fail  to  appreciate  the 
goals  or  thoroughly  practice  the  policies 
of  the  program. 

The  Centaur  Program  has  found  that 
the  most  effective  way  of  developing  the 
team  concept  at  the  subcontractor  or  ven¬ 
dor  is  through  visits  by  government  and 
contractor  representatives.  During  these 
visits,  the  representatives  inform  the 
subcontractor  or  vendor  of  the  current 
developments  in  the  program  and  instill 
in  their  personnel  the  idea  that  they  are 
making  a  major  contribution  to  the  success 
of  the  program  -  which  they  are.  Usually, 
after  a  visit,  a  letter  is  sent  with  pho¬ 
tographs  of  Important  developments  so  it 
can  be  posted  at  an  appropriate  place  to 
try  to  contact  as  wide  a  spectrum  of  em¬ 
ployees  at  the  vendor  or  subcontractor 
plant  as  possible. 

Problem  awareness  and  solving  have 
contributed  tremendously  to  mission  as¬ 
surance  on  the  Centaur  Program.  Although 
the  problems  on  the  program  have  occurred 
in  every  technical  discipline  and  at 
various  levels  of  seriousness,  the  manner 
in  which  they  are  brought  to  management's 
attention,  assigned  a  priority,  and  closed 
out  is  similar  and  formal. 

Problem  awareness  in  the  Centaur 
Program  begins  with  providing  the  neces¬ 
sary  checks  at  the  contractor  and  associ¬ 
ate  contractor  to  insure  that  all  fail¬ 
ures  are  detected.  The  program  uses 
prime  contractor  and  associate  contrac¬ 
tor  quality  personnel,  resident  engineers, 
dally  contact  between  the  program's  sys¬ 
tem  engineers,  and  personnel  of  delegated 
government  agencies  (DCAS)  to  surface 


failures.  With  such  a  system,  a  test 
falure  or  receiving  Inspection  (subcon¬ 
tractor  or  vendor)  problem  Is  quickly 
located  and  the  subsequent  analysis  re¬ 
ceives  adequate  and  expert  attention.  To 
detect  problems  occurring  at  a  subcontrac¬ 
tor  or  vendor,  the  program  relies  on  con¬ 
tractor  source  Inspectors  as  well  as  DCAS. 
Historically,  the  program  has  found  It 
more  difficult  to  surface  these  problems. 
However,  it  has  found  that  certain  signs 
such  as  rescheduling  of  delivery  dates  or 
lack  of  cooperation  usually  are  Indica¬ 
tive  of  a  problem  with  the  product  and, 
therefore,  a  visit  to  the  subcontractor 
or  vendor  should  be  arranged.  An  extremely 
important  aspect  of  problem  awareness  is 
maintaining  good  failure  records.  Such 
records  make  it  easy  to  assess  whether 
failures  are  generic;  i.e.,  failures  that 
possess  a  common  link. 

Many  of  the  test  failures  occurring 
on  the  Centaur  Program  are  random  in  na¬ 
ture.  For  instance,  a  piece  of  electro¬ 
nic  hardware  may  fall  because  of  a  diffu¬ 
sion  defect  in  a  transistor.  Such  a 
phenomenon  is  possibly  the  result  of  a 
piece  of  dirt  findings  its  way  into  the 
processing  of  the  transistor’s  wafer 
which  is  a  random  occurrence  and  is  not 
representative  of  other  transistors  fabri¬ 
cated  by  the  vendor  unless  many  of  the 
vendor's  transistors  fail  as  a  result  of 
diffusion  defects. 

At  certain  times,  however,  the  pro¬ 
gram  has  found  that  a  certain  group  of 
failures  possess  a  common  link  and,  there¬ 
fore,  are  generic.  Whenever  such  a  link 
exists  between  failures,  steps  must  be 
quickly  taken  to: 

1.  Determine  the  cause  of  failure 

2.  Bound  the  problem 

3.  Develop  effective  corrective 
action 

To  insure  that  the  above  activities  are 
carried  out  adequately,  the  problem  is 
elevated  by  management.  Elevation  allows 
management  to  track  the  progress  on  bound¬ 
ing  and  solving  the  problem  and  allows 
them  to  assure  the  corrective  action  is  in 
concert  with  the  overall  program  mission 
assurance  philosophy  and  goals. 


To  determine  the  cause  of  failure 
and  also  bound  the  problem,  the  program 
uses  all  avenues  available  to  it  which 
include: 

1.  Sophisticated  failure  analysis 
techniques  and  tools 

2.  Analysis  of  relevant  system  and 
component  tests 

3.  Similar  component  analysis 

4.  Subcontractor  or  vendor  visits 

to  review  component  processing  and  materi¬ 
als 

The  goal  of  the  above  activities  is  to 
obtain  mutually  supportive  data  to  deter¬ 
mine  the  cause  of  the  problem,  define  its 
boundaries,  and  develop  effective  cor¬ 
rective  action.  Engineers  involved  in  a 
particular  problem  report  to  the  Centaur 
Program  management  as  frequently  as  daily 
if  the  problem  has  much  urgency  or  visi¬ 
bility  associated  with  it,  or  as  infre¬ 
quently  as  weekly  if  the  problem  is 
rather  minor.  The  program  does,  however, 
conduct  open  Problem  Review  meetings  and 
closed  Problem  Review  meetings  approxi¬ 
mately  monthly  to  insure  all  problems  are 
being  or  have  been  adequately  solved. 
During  these  reviews,  subcontractor  and 
vendor  problems  are  surfaced  and  steps 
for  their  resolution  planned.  Finally, 
the  investigations  that  resolved  the  prob¬ 
lem  and  corrective  actions  are  documented 
in  a  problem  closeout  memorandum.  The 
program  will  not  conduct  a  launch  unless 
all  problems  associated  with  the  vehicle 
to  be  launched  are  close''. 

The  NASA  program  personnel  insure 
that  failure  information  is  transmitted 
between  the  prime  and  associate  contrac¬ 
tors  to  assure  that  effects  of  the  prob¬ 
lem  on  the  overall  program  or  vehicle  can 
be  evaluated.  The  program,  through  its 
reliability  engineers,  inputs  failure 
and  problem  data  into  the  Government 
Industry  Data  Exchange  Program  (GIDEP) 
and  monitors  it  to  assess  the  latest 
ALERT  data's  effect  on  the  Centaur 
Program,  Finally,  the  program's  engine¬ 
ers  and  management  are  in  frequent  con¬ 
tact  with  other  NASA  programs  to  inquire 
about  any  problems  or  concerns  they  are 
Invovled  with. 


Whenever  a  failure  occurs  during  a 
launch,  regardless  of  whether  the  failure 
is  of  a  generic  nature  or  not,  a  very 
formal  resolution  is  necessary.  This  is, 
of  course,  a  result  of  the  visibility  of 
such  a  failure.  Under  such  conditions, 
the  steps  of  the  flight  failure  investiga¬ 
tion  methodology  are  listed  below: 

1.  Establish  Failure  Investigation 
Teams 

a .  NASA/LeRC 

b.  Contractors 

2.  Establish  NASA  Review  Board 

3.  Analyze  flight  data 

A.  Recover  and  analyze  flight  hard¬ 
ware 

5.  Develop  failure  tree 

6.  Develop  possible  failure  scenarios 

7.  Formulate  and  perform  investiga¬ 
tive  tests 

8.  Consult  other  experts 

a.  Within  NASA 

b.  Nationally  known  authorities 

9.  Define  the  most  probable  cause  of 
the  failure 

10.  Initiate  corrective  actions 

11.  Respond  to  the  NASA  Review  Boara 
recommendations 

12.  Report  to  NASA  Headquarters  on 
findings 

13.  Finalize  and  Implement  corrective 
actions 

III.  PROGRAM  experiences  AND  THEIR  IMPACT 
ON  MISSION  ASSURANCE  SYSTEMS 


In  order  to  understand  more  fully  how 
program  experiences  have  affected  the 
Centaur  mission  assurance  system,  the  over¬ 
all  experience  of  the  program  with  regard 
to  mission  assurance  will  first  be  dis¬ 
cussed.  This  discussion  is  based  on  a 
categorization  of  flight  readiness  prob¬ 
lems  during  the  period  from  early  1977  to 


late  1978,  a  period  of  1  1/2  years. 

Flight  readiness  problems  are  defined  as 
those  discussed  with  top  LeRC  management, 
payload  customer,  and  NASA  Headquarters 
personnel.  During  this  1  1/2-year  period, 
A6  flight  readiness  problems  were  pre¬ 
sented.  Seventeen  problems  were  associ¬ 
ated  with  mechanical  systems,  whereas  29 
problems  were  associated  with  avionics 
systems.  Our  conclusion  is  that  this 
mix  represents  the  relative  complexity 
(high  part  density  with  associated  single 
point  failure  modes) of  avionics  systems 
over  mechanical  systems.  If  one  were  to 
classify  these  A6  problems  according  to 
where  they  were  found,  one  would  get  the 
distribution  indicated  in  Table  1: 

This  type  of  classification  indicates 
that,  because  the  avionics  system  is  more 
readily  checkable  than  the  mechanical 
system  and  can  operate  before  flight  as 
an  Integrated  system,  more  avionics  prob¬ 
lems  are  found  before  flight.  A  corol¬ 
lary  of  this  fact  is  that,  because  the 
numerous  mechanical  components  (l.e., 
propulsion  systems,  structures,  etc.) 
operate  as  an  Integrated  system  only  in 
flight,  more  mechanical  problems  are 
found  in  flight. 

This  justifies  the  Centaur  Program 
utilization  of  flight  simulation  techni¬ 
ques  as  extensively  as  possible  during 
ground  checkout  and  adequate  instrumenta¬ 
tion  to  evaluate  flight  problems.  It 
should  be  pointed  out  that  all  flight 
readiness  review  problems  in  Table  1  did 
not  affect  the  Success  of  the  launch.  In 
fact,  only  one  of  the  13  in-flight  prob¬ 
lems  in  Table  1  did  affect  the  flight 
success. 

If  one  classified  this  same  group  of 
problems  according  to  cause,  the  result 
would  be  that  shown  in  Table  II. 

This  classification  reveals  that  the 
manner  in  which  hardware  is  processed 
contributed  to  a  majority  of  program  prob¬ 
lems.  This  data  provides  the  justifica¬ 
tion  for  the  program  expending  much  effort 
to  thoroughly  and  periodically  review  con¬ 
tractor,  subcontractor,  and  vendor  fabri¬ 
cation  procedures  (through  some  formal 
audit  program)  in  order  to  reduce  process 
caused  problems.  These  reviews  or  audits 
are  usually  performed  by  contractor/gov- 
emraent  teams.  The  members  of  the  teams 


are  experienced  In  the  fabrication  and 
quality  procedures  associated  with  a 
particular  piece  of  hardweare.  During  a 
particular  audit  the  team  will  review  sub¬ 
contractor/vendor  activities  concerning  a 
particular  piece  of  hardware  such  as  the 
overall  fabrication  process,  critical  pro¬ 
cesses,  quality  practices,  and  adequacy 
of  hardware  records. 

This  same  group  of  problems  was  final¬ 
ly  classified  in  accordance  with  who  was 
mainly  responsible  for  the  cause.  The 
results  of  this  classification  are  shown 
in  Table  III. 

Table  HI  indicates  that  vendors  con¬ 
tribute  to  a  substantial  proportion  of  the 
total  problems.  This  justifies  the  pro¬ 
gram's  attempt  to  apply  its  mission  as¬ 
surance  system  to  the  vendors  as  well  as 
the  contractors  (associate  and  prime). 

Placing  problems  into  categories  as 
pointed  out  above  is  helpful  in  evaluating 
the  program's  mission  assurance  activities 
as  well  as  pointing  out  to  management 
where  additional  effort  should  be  exer¬ 
cised.  An  example  of  this  is  the  deci¬ 
sion  by  the  program's  management  to  con¬ 
duct  an  electronic  parts  Improvement  pro¬ 
gram.  This  is  an  example  of  how  a  chang¬ 
ing  environment  external  to  the  program 
made  it  necessary  to  adjust  the  program's 
mission  assurance  system.  In  this  parti¬ 
cular  case,  the  changing  environment  was 
the  the  change  in  the  major  customer  of 
the  microelectronics  Industry  from  the 
aerospace  user  to  the  commercial  user, 
and  the  Industry  changing  from  fabrica¬ 
ting  high  reliability  devices  on  custom 
lines  to  fabricating  these  devices  with 
mass  production  techniques. 

It  was  concluded  as  a  result  of  clas¬ 
sification  of  major  program  problems 
during  the  mid  70's  that  electronic  piece 
parts  were  causing  an  inordinately  large 
percentage  of  the  important  problems. 

For  Instance,  in  1975  about  30  percent  of 
of  flight  readiness  problems  were  due  to 
electronic  piece  parts.  Furthermore, 
this  percentage  seemed  to  be  on  the  in¬ 
crease.  The  program  management  decided 
to  change  the  mission  assurance  system  in 
this  area  to  hopefully  reverse  this  trend. 
At  this  time,  the  most  significant  mission 
assurance  characteristic  of  electronic 
parts  was  to  procure  them  using,  where- 


ever  possible,  specifications  and  vendor 
checks  developed  from  the  manned  space 
program. 

The  first  phase  of  this  activity 
consisted  of  forming  an  electronic  parts 
improvement  task  force  in  the  fall  of 
1975.  Its  objective  was  to  evaluate 
electronic  parts  activities  of  the  pro¬ 
gram's  contractors  who  use  relatively 
large  amounts  of  electronic  piece  parts. 
The  task  force  membership  consisted  of 
electronic  piece  part  engineers  from 
LeRC,  GDC,  Teledyne,  Honeywell,  George  C. 
Marshall  Space  Flight  Center  (NASA's 
lead  Center  for  electronic  piece  parts) 
and  Aerospace  Corporation.  The  task 
force's  method  of  operation  was  to  first 
bring  the  contractors  together  to  have 
open  exchanges  on  their  experiences  in 
all  their  electronic  piece  parts  activi¬ 
ties,  and  secondly,  to  visit  many  of  the 
program  vendors  to  inform  them  of  the 
program's  objectives  and  see  how  within 
the  framework  of  their  operation  these 
objectives  could  be  satisfied. 

The  task  force  concluded  that  the  con¬ 
tractors  should  strive  for  commonality  in 
parts  activities  such  as  parts  usage, 
specfication,  special  testing,  and  ven¬ 
dors.  Secondly,  the  task  force  concluded 
that  the  contractor  source  inspection 
should  be  improved.  It  felt  this  could 
be  accomplished  by  sending  source  inspec¬ 
tors  to  the  contractors’  home  plant  more 
often  for  more  contact  with  the  parts 
engineers  in  an  environment  where  he 
could  learn  more  about  the  parts  he  is 
involved  with,  problems  encountered  with 
these  parts,  and  changes  in  methods  of 
testing  parts.  Visits  to  the  vendors  re¬ 
vealed  that  any  of  the  program's  vendors 
were  capable  of  fabricating  a  good  part 
at  any  given  time  or  on  the  other  hand, 
a  bad  part  at  any  given  time.  This 
meant  better  parts  could  not  be  obtained 
by  changing  or  eliminating  vendors,  but 
might  be  obtained  by  better  monitoring  of 
the  particular  lot  of  parts  procured. 

Tile  task  force  also  found  that  special 
NASA  surveillance  of  vendors  was  rapidly 
declining.  The  program  in  the  past  had 
relied  heavily  on  this  surveillance  which 
was  the  result  of  the  manned  space  pro¬ 
grams,  which  by  this  time  (about  1975) 
were  nearing  completion.  The  final  major 
finding  of  the  task  force  was  that  re¬ 
quiring  special  fabrication  by  the  vendor 
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was  undesirable.  Special  fabrication 
usually  meant  a  part  had  to  be  fabricated 
on  a  less  capable  line  or  at  least  taken 
from  the  more  capable  lines  for  special 
processing  which  meant  its  quality  could 
be  reduced  due  to  this  type  of  processing. 

As  a  result  of  these  task  force  find¬ 
ings,  the  program  decided  , to  direct  the 
contractors  to  convert  wherever  possible 
to  NASA  standard  devices  (MIL-STD-975) 
and  to  further  increase  the  quality  of 
these  devices  by  conducting  a  rescreenlng 
of  these  parts  at  a  preselected  screening 
laboratory.  The  program  felt  this  would 
promote  part  specification  and  testing 
commonality  among  contractors.  The  pro¬ 
gram  also  decided  through  specification 
and  monitoring  to  tighten  its  control  on 
part  fabrication  wherever  it  was  consis¬ 
tent  with  activities  at  the  vendor  fabri¬ 
cation  line.  For  example,  the  program 
required  the  fabrication  of  all  its  parts 
be  done  in  this  country  and  sample  inter¬ 
nal  lead  bond  pull  test  be  conducted  on 
each  lot  of  parts.  In  addition  to  con¬ 
trols  such  as  these,  the  program  decided 
to  upgrade  source  inspection  in  order  to 
Increase  its  control  on  part  fabrication. 

In  order  to  monitor  vendor  capability 
and  effectiveness  of  source  inspection 
and  to  detect  gross  part  inconsistencies, 
the  program  required  device  construction 
data  sheets  be  supplied  with  the  devices 
procured,  and  a  destructive  physical  anal¬ 
ysis  be  conducted  by  the  contractor  or  a 
designated  laboratory  on  a  sample  of  the 
vendor's  lot  of  devices  procured.  This 
also  Insured  that  any  changes  in  cons¬ 
truction  or  gross  deficiencies  could  be 
detected  and  evaluated  early.  Finally, 
the  program  decided  to  create  a  Launch 
Vehicle  Electronic  Parts  Steering 
Committee  to  monitor  all  electronic  parts 
activities  such  as  rescreening  and  results 
of  the  destructive  physical  analysis. 

These  efforts  had  a  very  noticeable  effect 
on  the  quality  of  electronic  parts  by 
1978.  In  that  year,  there  were  21  flight 
readiness  problems.  However,  only  one  of 
these  could  be  classified  as  an  electronic 
piece  part  problem. 

Sometimes  a  program  can  improve  its 
mission  assurance  system  by  taking  advan¬ 
tage  of  unique  capabilities  and  experience 
that  exist  within  its  parent  organization. 
This  turned  out  to  be  the  case  with  the 


Centaur  implementation  of  the  microelec¬ 
tronic  hybrid  parylene  passivation  pro¬ 
gram.  This  activity  has  visibly  improved 
the  Centaur  Program  mission  assurance  as 
well  as  other  high  reliability  programs 
both  within  NASA  and  private  industry. 

The  problem  which  gave  birth  to  this 
activity  was  numerous  acceptance  testing 
failures  in  the  launch  vehicle  digital 
computers  caused  by  conductive  particles 
in  hybrid  microcircuits.  These  failures 
were  the  result  of  (1)  a  lack  of  glasslva- 
tion  of  the  hybrid  microcircuit  die,  (2) 
improved  methods  of  vibrating  the  computer 
as  well  as  techniques  for  detecting  a 
failure  and  (3)  package  design  that  made 
it  difficult  to  control  solder  particles 
resulting  from  the  hybrid  cover  seal  pro¬ 
cess.  Changing  any  one  of  these  items  to 
reduce  the  failures  would  either  compro¬ 
mise  quality  or  seriously  Impact  the  pro¬ 
gram  both  from  a  cost  and  schedule  stand¬ 
point. 

To  resolve  this  problem,  the  Centaur 
Program  initiated  an  activity  aimed  at 
passivating  the  hybrid  microcircuits  by 
using  a  coating  material  called  parylene. 
The  parylene  coating  material  was  chosen 
mainly  because  its  important  characteris¬ 
tics,  shown  in  Table  IV,  applied  favor¬ 
ably  to  the  hybrid  microcircuits  in  ques¬ 
tion. 

The  above  characteristics  coupled  with 
unique  in-house  (LeRC)  expertise  made 
parylene  especially  attractive  in  this 
application. 

Since  the  vendor  who  fabricated  the 
hybrid  microcircuits  had  no  experience  in 
the  parylene  process,  a  coater  had  to  be 
installed  at  his  facility.  After  coater 
installation,  hybrid  microcircuit  tooling 
and  processing  had  to  be  developed  under 
the  direction  of  government  program  and 
parylene  engineers.  Also,  tests  were  con¬ 
ducted  to  determine  if  parylene  or  the 
manner  in  which  it  was  being  applied  had 
any  effect  on  the  hybrid  microcircuits. 

Once  the  above  activities  were  satis¬ 
factorily  completed,  qualification  testing 
was  began  at  the  part  level  to  satisfy  the 
program  requirement;  that  neither  unquali¬ 
fied  parts  or  processes  shall  be  used  on 
the  vehicle.  This  qualification  program 
was  patterned  after  the  standard  military 
qualification  for  microelectronic  circuits. 


except  operating  life  tests  and  high 
temperature  storage  were  conducted  for 
2,000  hours  rather  than  the  usual  1,000 
hours . 

The  parylene  passivation  program  has 
greatly  improved  the  quality  of  the  launch 
vehicle  digital  computer.  Since  its  im¬ 
plementation,  no  hybrid  microcircuits 
coated  with  parylene  have  had  a  particle 
induced  failure;  and  since  it  was  speci¬ 
ally  designed  for  hybrid  microcircuits 
which  caused  digital  computer  vibration 
failures,  a  vibration  failure  now  is  a 
rare  occurrence.  Furthermore,  because 
particle  failures  are  usually  intermit¬ 
tent,  failure  analysis  results  were  not 
always  conclusive  and  large  amounts  of 
hardware  occasionally  had  to  be  declared 
nonflight  -  a  costly  and  schedule  im¬ 
pacting  alternative.  With  the  implemen¬ 
tation  of  parylene,  this  no  longer  occurs. 

Thus  far,  I  have  discussed  techniques 
and  activities  implemented  on  a  high 
reliability  program  for  the  sole  purpose 
of  mission  assurance;  i.e.,  preventing  a 
failure  which  causes  the  program  to  not 
meet  its  primary  mission. 

At  times,  such  efforts  do  not  prevent 
this  from  happening  and  the  Centaur 
Program  is  no  exception.  How  a  program 
reacts  to  such  a  serious  occurrence  and 
recovers,  and  defines  the  "lesson  learned" 
corrective  action,  is  certainly  a  subject 
which  should  be  addressed. 

My  example  will  be  the  most  recent 
Atlas/Centaur  failure.  On  September  29, 
1979,  an  Atlas/Centaur  (AC-A3)  was 
launched  with  the  objective  of  placing 
an  INTELSAT  IV-A  communication  satellite 
into  a  geostationary  orbit.  The  flight 
was  terminated  55  seconds  into  flight 
when  the  vehicle  self-destructed.  From 
initial  reviews  of  the  data,  failure 
scenarios  were  developed  and  analysis, 
tests  and  data  reviews  were  immediately 
conducted  to  evaluate  these  potential' 
failure  scenarios. 

The  failure  analysis  was  significant¬ 
ly  aided  by  the  fact  that  the  vehicle 
was  destroyed  so  early  in  flight.  It  had 
not  traveled  far  downrange  and  much  of 
the  hardware  from  the  booster  section 
which  the  failure  scenarios  were  centered 
around  could  be  recovered  from  the 


Atlantic  Ocean.  Concurrent  with  recovery 
of  booster  hardware,  examinations  and 
supporting  tests  conducted  at  GDC  indica¬ 
ted  that  the  area  nearest  the  booster  gas 
generator  was  exposed  to  the  highest  tem¬ 
peratures.  The  recovered  gas  generator 
was  delivered  to  Rocketdyne,  the  associate 
contractor  responsible  for  the  booster 
engines,  and  detailed  testing  and  examina¬ 
tion  revealed  only  one  leak  source  in  the 
gas  generator  system,  this  being  a  crack 
in  a  brazed  joint  in  the  gas  generator 
outlet  manifold  assembly. 

Review  of  the  paper  history  assolcated 
with  the  gas  generator  manifold  assembly 
revealed  it  was  procured  as  one  of  a  last 
order  of  nine  items  from  a  vendor  of 
Rocketdyne.  Furtehrmore,  it  was  learned 
that  the  brazing  of  the  joint  was  per¬ 
formed  by  a  second  vendor  in  1971. 

This  second  tier  vendor  has  since 
gone  out  of  business,  but  interviewing  the 
the  ex-owner  indicated  that: 

1.  Braze  furnace  atmosphere  was  hy¬ 
drogen. 

2.  Carbon  tooling  was  not  disallowed 
for  brazing. 

3.  Carbon  tooling  was  frequently  used 
in  hydrogen  atmosphere. 

4.  Parts  were  received  precleaned. 

As  a  result  of  the  data  accumulated 
above  and  subsequent  matallurglcal  anal¬ 
ysis  of  the  failed  joint,  it  was  concluded 
that: 

1.  The  joint  was  carburized  and 
sensitized, 

2.  Extended  exposure  to  corrosive 
environment  at  ETR  had  occurred. 

3.  Fracture  was  a  result  of  inter¬ 
granular  corrosion. 

4.  Carburization  and  sensitization 
were  most  likely  caused  by  a  contaminated 
brazing  process. 

As  a  result  of  this  failure  and  subse¬ 
quent  analysis,  the  program  Implemented 
the  following  corrective  actions: 
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1.  Conducted  a  review  of  vehicle 
hardware  material  processing. 

2.  Developed  a  nondestructive  test 
to  identify  sensitized  hot  gas  manifold 
braze  joints. 

3.  Changed  contractor/vendor  specifi¬ 
cation  to  eliminate  use  of  carbon  fix¬ 
tures  . 

In  conclusion,  it  is  important  to 
note  that  even  a  program  operating  sue 
cessfully  several  years  with  formal  well- 
developed  mission  assurance  practices  is 
vulnerable  to  a  failure.  Therefore,  it 
must  continually  be  vigilant  to  detect 
the  quality  defects  that  exist  in  any 
system  and  be  ready  to  improve  its  mission 
assurance  system  as  a  result.  In  this 
case,  the  Centaru  Program  decided  to: 

1.  Review  and  improve  process  con¬ 
trols  at  critical  sub-tier  vendors. 

2.  Periodically  certify  critical 
sub-tier  vendors. 

3.  Use  coupon  samples  to  monitor 
critical  processing. 

In  spite  of  the  seriousness  of  this 
type  of  failure,  its  visibility,  and  its 
source  of  embarrassment  to  the  program, 
it  is  comforting  to  be  part  of  such  an 
effective  team  effort.  After  the  failure, 
the  Centaur  team  (government,  contractor, 
and  vendor)  worked  together  in  a  very  co¬ 
operative  and  efficient  manner  to  resolve 
the  problem.  Since  this  failure,  10  suc¬ 
cessful  Atlas/Centaur  launches  have  ocur- 
red.  This  is  an  excellent  testimony  that 
the  program  has  successfully  recovered 
from  the  failure. 

IV.  CONCLUSION 

An  effective  program  mission  assur¬ 
ance  system  must  have  a  strong  base  which 
the  Centaur  Program  has  in  its  Mission 
Assurance  Plan.  As  pointed  out  herein  an 
effective  plan  must  be  dynamic  and,  there¬ 
fore,  must  provide  within  its  basic  plan 
provisions  for  change,  through  mission 
assurance  Improvement  activities.  In 
this  paper,  we  have  discussed  some  of  the 
more  prominent  recent  activities  which 
have  improved  the  mission  assurance  of  the 
Centaur  Program.  Although  these 


activities  were  initiated  to  meet  widely 
different  program  needs,  were  quite 
different  in  the  nature  of  their  work, 
involved  different  disciplines,  contrac¬ 
tors  and  vendors,  they  do  have  the  follow¬ 
ing  in  common: 

1.  A  customer  (NASA-LeRC)  familiar 
with  its  hardware  and  able  to  Identify  a 
specific  program  goal  to  better  meet  the 
objectives  of  the  program. 

2.  Contractors,  subcontractors,  and 
vendors  who  understand  and  appreciate  the 
objectives  of  the  program  and  have  a 
strong  desire  to  cooperate  with  the  Govern¬ 
ment  and  each  other  to  satisfy  a  specific 
goal. 

3.  An  existing  mission  assurance 
system  which  encourages  periodic  assess¬ 
ment  of ; 

a.  subcontractors  and  vendors 

b.  hardware  improvements  and 

changes 

and  allows  changes  to  the  program  associ¬ 
ated  with  these  items  after  they  have 
been  adequately  evaluated. 

These  characteristics  have  been  essential 
to  the  Centaur  mission  assurance  system 
and  its  continuous  improvement.  Without 
such  an  effective  mission  assurance  sys¬ 
tem,  the  Centaur  Program  would  not  be  the 
mature  and  successful  program  that  it  is 
today. 


TABLE  I 


WHERE  FLIGHT  READINESS  PROBLEMS  ARE  FOUND 


WHERE 

^"v^UND 

FACTORY 

LAUNCH 

IN 

SYSTEM 

PAD 

FLIGHT 

Mechanical 

2 

7 

8 

Avionics 

16 

10 

3 
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FLIGHT  READINESS  PROBLEM  CAUSE 


CAUSE 

TEST 

PROCESS 

MATERIAL 

DESIGN 

OTHER 

SYSTEM 

Mechanical 

2 

8 

2 

2 

3 

Avionics 

2 

15 

0 

4 

8 

TABLE  III 


FLIGHT  READINESS  PROBLEM  RESPONSIBILITY 


- - - 

■■■■ 

SYSTEM  — ^ 

CONTRACTOR 

UNKOWN 

Mechanical 

6 

10 

1 

Avionics 

11 

17 

1 

Includes  subcontractors 

TABLE  IV 

IMPORTANT  PARYLENE  CHARACTERISTICS 


Thin  Uniform  Coating  Thickness 

Resistant  to  Microelectronic  Processing  Chemicals 

Mechanical  and  Electrical  Properties  Acceptable  to  Hybrid 
Microci rcui ts 

Stable  in  Inert  Atmosphere 

Application  Relatively  Simple  and  Quality  Easily  Controlled 


An  Overview  of  Space  Division 
Part  Management  and  Technical 
Requi rements 

Sgt  Clifford  Schroeder 
USAF  Space  Division 

Space  Division  Is  developing  and  Improv¬ 
ing  the  requirements  for  space  quality 
parts  which  are  neqded  In  order  to  achieve 
success  on  space  missions.  In  this  pre¬ 
sentation,  I  will  discuss  the  principal 
management  and  technical  requirements 
and  their  Implementation  at  the  prime  and 
subtler  contract  levels.  Subtler  con¬ 
tractors  Include  subcontractors,  suppliers, 
and  vendors. 

Unique  Features  of  Space  Programs 

1.  Long  mission  life  -  10  year  goal 

2.  No  maintenance  In  space 

3.  Complex,  high-cost  equipment 

4.  Use  advanced  technology 

5.  High-cost  development  tests  and  models 

Most  of  you  are  familiar  with  the  unique 
features  of  space  programs,  but  I  would 
like  to  review  them  briefly.  The  new 
STRATSAT  and  DSCS  III  programs  have  a 
design  life  of  10  years  with  a  required 
life  of  7  years.  No  operational  mainte¬ 
nance  Is  available  In  space  -  yet.  There¬ 
fore,  we  must  build  them  right  the  first 
time.  Spacecraft  have  numerous  subsystems 
and  components  using  new  complex  piece 
parts  and  exotic  materials.  The  vehicle's 
parts  count  reflects  their  Increased 
complexity.  IDCSP,  a  forerunner  of  the 
DSCS  spacecraft  had  4700  parts  per  space¬ 
craft.  DSCS  II  Increased  to  36,000  parts, 
and  the  current  DSCS  III  anticipates 
150,000  parts.  Although  some  of  this 
Increase  Is  to  provide  redundancy  for 
reliability  Improvement,  the  trend  to 
Increase  complexity  Is  clear.  The  cost 
for  one  spacecraft  and  launch  vehicle  Is 
on  the  order  of  $100  million. 


Achieving  additional  performance  Is  usually 
a  program  objective  -  more  channels,  a 
larger  number  of  functions,  and  more  auton¬ 
omy  requiring  greater  on-board  data  pro¬ 
cessing  capability.  By  advanced  technology 
relative  to  electronic  piece  parts,  I  mean 
microprocessors,  complex  hybrid  circuits, 
large  scale  and  very  large  scale  Integrated 
circuits  (VLSI).  These  types  of  parts  need 
high  reliability  and  strong  management 
requirements. 

Engineering,  breadboard,  and  development 
models  and  tests  also  are  becoming  Increas¬ 
ingly  complex  and  expensive.  In  some 
Instances,  we  have  had  a  quarter  of  a 
million  dollars  Invested  1n  one  develop¬ 
ment  model  and  test. 

Elements  of  the  Parts  Problem 

1.  Long  mission  life,  complex  equipment, 
requires  lower  part  failure  rates  to  achieve 
mission  reliability. 

2.  Piece  parts  can  be  single  point  failure 
modes. 

3.  Purchase  of  "space"  electronic  parts 
Is  difficult. 

4.  Parts  failing  during  assembly  and  test 
Increase  costs,  delay  schedules.  Indicate 
low  reliability  of  equipment. 

5.  Part  failures  have  caused  significant 
mission  and  ground  equipment  failures. 

The  constraints  and  requirements  at  the 
system  level  are  reflected  In  the  require¬ 
ment  down  to  the  piece  part  level. 

When  the  mission  life  and  parts  count  are 
both  doubled,  the  parts  failure  rate  must 
be  reduced  to  one  fourth  of  Its  value  to 
achieve  the  same  mission  reliability.  If 
the  reliability  Is  also  Increased,  a  further 
improvement  In  the  parts  failure  rate  Is 
needed. 

Single  point  failure  modes  can  be  allevia¬ 
ted  with  redundancy  but  redundancy  alone 
won't  achieve  mission  success. 

Some  of  the  purchasing  difficulties  experi¬ 
enced  are  no  qualified  source  from  which 
to  purchase  a  MIL  Spec  part,  nonresponse 
by  vendors  to  purchase  requests,  long  lead 
times— on  the  order  of  15  to  18  months— 
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and  low  quality  parts.  We  recognize  that 
purchasing  is  difficult,  and  are  working 
on  Class  S  procedures  and  surveillance 
requirements  to  alleviate  those  diffi¬ 
culties. 

In  addition  to  the  repair  work  and  replace¬ 
ment  part,  the  actual  cost  of  part  failures 
during  assembly  Includes  the  work  by  the 
quality  inspectors,  material  review  board, 
corrective  action  board,  and  administra¬ 
tive  overhead.  Many  programs  don't 
investigate  the  part  failures  during  equip¬ 
ment  fabrication,  but  sometimes  these 
failures  were  indications  of  a  hardware 
problem. 

Mission  failures  attributed  to  piece  part 
failures  were  described  by  Col  Schlosser 
and  others  at  the  1978  Mission  Assurance 
Workshop.  Last  year,  at  least  one  ABRES 
flight  test  anomaly  was  attributed  to  a 
piece  part  failure. 

SD  Approach  to  Improving  Space  Parts 
Reliability  and  Availabili^ 

1.  Comprehensive  part  management  program 
for  all  parts. 

2.  Standard  "space  quality"  technical 
requirements. 

3.  Identification/prevention  of  all  part 
failure  causes. 

4.  Development  of  Class  "S"  (space)  MIL 
Standard  parts. 

5.  Space  part  coordinated  procurement  by 
prime  contractors. 

The  recommendations  of  the  parts  panel  at 
the  1978  workshop  have  been  a  strong 
influence  on  Space  Division's  approach  to 
improving  the  reliability  and  availability 
of  parts  for  use  in  space  systems.  This 
approach  includes  extensive  management  by 
each  program  and  prime  contractor  of  all 
parts  used  in  the  space  hardware,  as  well 
as  developing  and  implementing  "space 
quality"  technical  requirements  for  all 
standard  and  nonstandard  parts.  These 
requirements  include  standardized  test 
methods,  the  identification  of  all  known 
causes  of  parts  failures,  and  any  known 
methods  of  preventing  them,  such  as  scanning 
electron  microscope  (SEM)  inspection  and 
wafer  lot  tests. 


When  buying  small  quantities  of  parts, 
as  is  typical  on  space  programs,  coordi¬ 
nated  procurement  has  demonstrated  advantages 
for  obtaining  high  reliability  parts— 
less  documentation,  better  schedules,  and 
f«i#er  samples  required  for  destructive 
testing. 

Documented  Parts  Program  Requirements 

1.  SAMSO-STD  73-2C 

a.  Initial  standard  for  space  parts 

b.  Management,  technical  requirements 

2.  MIL-STD-1546  -  Management  Program  for 
Space  Parts. 

3.  MIL-STD-1547  -  Technical  Requirements 
for  Space  Parts. 

Some  management  practices  have  proven  more 
effective  than  others  for  obtaining  high 
quality  space  parts.  These  practices 
were  defined  and  documented  in  SAMSO-STD 
73-2C  to  provide  an  organized  structure 
for  parts  control  on  space  and  missile 
programs.  Those  requirements  have  been 
revised  to  include  better  technical  require¬ 
ments  and  management  techniques. 

Revision  of  SD  Parts  Standards 

1.  Part  program  management  (MIL-STD-1546). 

a.  Expansion  of  SAMSO-STD  73-2 

b.  Includes  AF  parts  control  require¬ 
ments  (MIL-STD-965) 

c.  Requires  comprehensive  parts 
management  by  contractor 

2.  Technical  (spec)  requirements  for 
contractor  parts  (MIL-STD-1547). 

a.  Section  for  each  part  type 

b.  Refers  to  space  quality  MIL  Spec 
requirements 

c.  Incorporates  lessons  learned/ 
failure  preventions 

The  management  requirements  are  written  in 
MIL-STD-1546,  the  technical  requirements 
for  the  parts  are  in  MIL-STD-1547.  These 
documents  are  scheduled  for  publication 
this  year  and  have  been  through  several 
coordination  cycles  with  the  Space  Parts 
Working  Group  which  includes  representatives 
from  Industry,  NASA,  and  the  military. 


.  • 
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3.  Application  and  design. 


We  don't  claim  to  have  developed  perfect 
requirements,  but  we  are  improving  them. 

MIL-STD-1546  -  Management  Requirements 

1.  Parts,  materials,  processes  control, 
and  standardization  program. 

2.  PMP  control  board. 

3.  PMP  program  plan. 

4.  Parts  selection  criteria. 

5.  Documentation  review/approval. 

6.  PMP  list. 

7.  Coordinated/centralized  procurement. 

8.  Supplier  control  and  surveillance. 

9.  MPCAG  participation. 

These  are  the  principle  requirements  of 
MIL-STD-1546.  I  will  expand  on  some  of 
them  later. 

Management  responsibilities  are  assigned 
to  the  PMP  control  board  to  provide  high 
visibility  into  all  parts  control  efforts. 

The  program  plan  describes  in  greater  detail 
than  the  military  standard  how  the  require¬ 
ments  will  be  implemented  and  satisfied. 

In  order  to  manage  the  parts  control  pro¬ 
gram,  the  parts  board  reviews  and  approves 
required  documentation  such  as  the  parts 
program  plan,  part  specifications,  non¬ 
standard  part  approval  requests,  and  reports 
on  schedules,  tests,  and  failures. 

The  functions  of  the  military  parts  control 
advisory  group  are  described  in  Appendix  E 
to  the  military  standard.  This  group  from 
the  Defense  Electronics  Supply  Center  (DESC) 
provides  advise  on  the  use  of  military 
standard  parts. 

MIL-STD-1547  -  Technical  Requirements 

1.  Interim  method  to  obtain  space  qualified 
parts  not  on  Class  "S"  QPL. 

2.  Limits  of  use  of  "unreliable"  part 
types . 


a.  Derating,  "end-of-life",  mounting 

4.  Part  design/construction. 

5.  Quality  assurance  provisions. 

a.  In-process  Inspections/controls 

b.  Group  A  screening  100% 

c.  Group  B  lot  conformance 

d.  Qualification  tests 

6.  Destructive  Physical  Analysis  (DPA). 

MIL-STD-1547  contains  the  technical  require¬ 
ments  to  be  put  in  the  specifications 
used  for  purchasing  the  parts. 

Some  application  requirements  for  the 
parts  use  in  the  end  item  are  included. 

The  "end-of-life"  requirement  enables  a 
circuit  to  perform  as  specified  until 
aging  of  the  parts  electrical  parameters 
exceed  the  tolerances  specified. 

The  requirements  were  compiled  using 
information  from  lessons  learned  and 
corrective  actions  on  failures.  These 
indicated  some  "unreliable"  part  types 
which  require  procuring  activity  approval 
for  use. 

Several  tests  which  are  done  on  a  sample 
basis  in  present  military  standards  have 
been  changed  to  100%  screens.  Qualifica¬ 
tion  tests  apply  to  a  part  type  and 
usually  reference  a  military  standard. 

Destructive  physical  analysis,  the  dissec¬ 
tion  and  examination  of  a  part,  is  an 
effective  method  to  detect  contamination, 
poor  workmanship,  and  design  changes.  It 
detects  semiconductor  device  problems  like 
counterfeits,  weak  bonds,  and  die  attach 
problems.  The  DPA  requirements  are 
documented  in  MIL-STD-1580  which  is  sched¬ 
uled  for  publication  this  year. 

The  need  for  strong  management  and  "failure 
free"  parts  does  not  lessen  at  subtler 
levels,  but  how  to  apply  or  implement  the 
requirements  may  change  as  the  scope  of 
the  efforts  become  more  specialized. 

Having  presented  a  brief  overview  of  SD 
parts  requirements,  the  remainder  of  the 
presentation  will  offer  some  examples  of 
application  of  these  requirements  at  the 
various  contract  levels. 


MIL-STD-1546 


and  semiconductor  devices. 


1.  Applies  in  total  to  prime  as  tailored 
by  SOW. 

2.  Applies  to  sub  as  tailored  by  SOW 
except  where  prime  takes  action  in  lieu 
of  sub,  e.g., 

a.  Manage  program  parts  list 

b.  Chair  PMPCB 

3.  Does  not  apply  directly  to  parts 
manufacturers. 

The  prime  contractor  becomes  responsible 
to  the  procuring  activity  for  satisfying 
the  management  requirements  of  MIL-STD- 
1546,  as  tailored  in  the  statement  of 
work.  Some  requirements  specify  functions 
to  be  performed  by  the  prime  contractor— 
for  example,  managing  the  program  parts 
list  and  chairing  the  PMPCB.  For  other 
requirements,  the  prime  may  delegate 
authority  to  the  subtler  contractors  which 
become  responsible  to  the  prime  contractor. 
That  delegation  does  not  change  the 
responsibilities  of  the  prime  contractor 
to  the  procuring  activity. 

Achieving  space  reliability  must  be  a 
cooperative  effort  with  all  contractors, 
subcontractors,  and  vendors  doing  their 
part  to  assure  standard  space  quality 
parts  are  selected,  specified,  and  procured 
to  standard,  space  quality  requirements. 

MIL-STD-1547 

1.  Applies  to  all  parts  (other  than  JAN 
branded.  Class  S  parts  from  SD  PPL). 

2.  Applies  to  subs  the  same  as  primes 
except  where  waived  by  PMPCB  and  procuring 
activity. 

3.  Applies  to  parts  manufacturers  ^ 
included  in  program  part  specs. 

The  management  requirements  of  1546  do  not 
apply  to  the  piece  part  manufacturers.  A 
different  set  applies  to  them— the  technical 
requirements  of  MIL-STD-1547.  These  require¬ 
ments  apply  to  all  contractor  specified 
parts  to  assure  these  non- standard"  parts, 
if  approved  by  the  PMPCB,  are  procured  to 
requirements  used  to  buy  the  MIL-Standard 
Class  S  and  JAN  S  parts.  In  its  present 
form,  MIL-STD-1547  applies  to  all  electronic 
part  types— relays,  inductors,  resistors, 
capacitors,  as  well  as  integrated  circuits 


Except  where  waived  by  the  procuring 
activity  and  PMPCB,  these  requirements  for 
space  quality  parts  apply  through  all 
subtler  contracting  levels  to  the  part 
manufacturer  who  satisfies  the  requirements 
as  specified  in  the  applicable  purchasing 
document. 


Prime  Contractor  Subtler  Contractor 

Plan,  establish.  Satisfy  PMP  control 

conduct  a  PMP  con-  requirements  imposed 

trol  program  by  contract  flowdown 

Responsible  for  Perform  surveillance 

supplier  surveillance  if  required 

Establish  operating  Provide  member  for 

procedure  to  be  PMPCB 

followed  by  the  PMPCB 

Identify  responsi-  Perform  functions 

bill  ties  and  func-  as  required 

tions  for  the  PMP 
effort 

Develop,  provide  a  Each  subtier  con- 

PMPCB  master  task  tractor  provide  a 

schedule  schedule  for  the 

subcontracted  effort 

The  above  list  identifies  some  of  the  major 
elements  of  the  PMP  control  program  and 
compares  the  responsibilities  of  the  prime 
and  subtler  contractors. 

For  example,  the  prime  plans,  establishes, 
and  conducts  the  program.  The  subtler 
contractor  satisfies  the  responsibilities 
described  in  the  subcontract. 

Also,  each  subtler  contractor  provides  a 
schedule  of  his  effort  enabling  the  prime 
to  develop  a  schedule  for  the  total  pro¬ 
gram. 

Parts,  Materials,  and  Processes  Control 
Board 

Prime  Contractor  Subtler  Contractor 
Establish  a  PMPCB 

Chair  the  PMPCB  Provide  member 


Parts,  Materials,  and  Processes  Control 
Program 
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Establish  operating  Comply  with  proce- 

procedure  dure 

Establish  program  Provide  Inputs  for 

PMPSL  PMPSL 

Evaluate  NSPARs,  Prepare,  submit 

deviations  NSPARs,  deviations 

Review  failure  Inform  PMPCB  of 

reports,  MRB  actions  parts  problems 


Purchase  to  military  speci¬ 
fication  and  MIL-STD-1547 
requirements 

If  needed:  Nonstandard  parts 

Require  NSPAR,  PMPCB  approval 
Require  parts  specification 
Purchase  to  MIL-STD-1547 
requirements 


Prime  Contractor 

Schedule,  coordinate, 
administer  PMPCB 
meetings 

Prepare  PMP  documen¬ 
tation  when  contract 
requires 


Subtler  Contractor 

Attend,  provide 
technical  support 

Same 


Appendix  C  to  MIL-STD-1546  describes  the 
selection  criteria  for  parts  used  In  space 
systems.  The  criteria  applies  equally  to 
the  prime  and  subtler  contractors.  The 
preferred  parts  list  Is  Appendix  D  to  MIL- 
STD-1546.  Until  It  Is  Issued,  the  current 
preferred  parts  list  Is  SD-STD  73-4B. 

Coordinated  Procurement 


Ensure  subtler 
contractors  support 
PMPCB 


PMP  Sel ectlonCrl terla 


Maximize  use  of 
standard  PMP, 
accomplish  supplier 
surveys  as  required 


Prime  Contractor 

Organize,  manage  CP 
program 


Establish  list  of  CP 
parts 

Negotiate  common 
purchasing  agreement 
with  PMPCB  approved 
suppliers 

Submit  documentation 
for  procuring  activ¬ 
ity  approval 


Subtler  Contractor 

Participate  as 
required  by  contract, 
supplemental  agree¬ 
ments 

Submit  part  types, 
quantity,  need  date 

Purchase  from  approved 
suppliers 


Provide  Information 
as  required 


Comparing  the  respective  parts  control 
board  responsibilities,  the  prime  chairs 
the  board,  the  subcontractor's  members 
participate.  The  subtler  contractors 
provide  Information  such  as  NSPARs  and 
deviations  for  the  prime  to  evaluate. 
Contractually  required  documentation  must 
be  prepared  by  both. 


Same  criteria  applies  to  prime  and  subtler 
contractors. 


Preferred:  Standard  parts 

Preferred  parts  list  -  Class 
"S",  JAN  S 

Alternate:  Substitute  Class  "S"  parts 

Space  Division  preferred 
parts  list 

Requires  NSPAR,  PMPCB 
approval 

Preferred  part  types 

Space  Division  preferred 
parts  list 
Require  NSPAR,  PMPCB 
approval 


Common  purchasing 
agreement 

Cost  allocation  plan 

Manage  Incoming  Perform  required 

Inspection  require-  Inspections  plus 

ments  additional  Inspec¬ 

tions  as  desired 
for  confidence 

Coordinated  procurement  Is  defined  as  the 
combined  effort  of  the  prime  contractor 
and  subtler  contractor  on  a  given  program 
to  purchase  parts  through  the  use  of 
common  specifications,  common  purchase 
agreements,  and  unified  management.  The 
Space  Division's  Commander's  Policy  of 
using  coordinated  procurement  was  another 
result  of  the  recomnendatlons  from  the 
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1978  Mission  Assurance  Workshop.  Most 
contractors  who  have  implemented  it  saw 
the  advantages  mentioned  earlier.  Addi¬ 
tional  advantages  include  less  inspection 
manpower  and  lower  costs.  The  current 
requirements  for  centralized  procurement 
are  documented  in  Notice  3  to  SD-STD  73-2C. 

The  prime  contractor  organizes  and  manages 
the  coordinated  procurement  program.  The 
subtier  contractors  participate  as  required 
in  the  subcontract  document  and  supple¬ 
menting  agreements.  The  subs  submit  informa¬ 
tion  on  the  parts  they  need  and  the  prime 
prepares  the  combined  list  of  coordina¬ 
ted  procurement  parts.  The  prime  contract¬ 
or  has  overall  management  responsibility 
for  incoming  inspection  requirements. 

The  subtier  contractors  perform  the 
inspections  required  and  they  may  perform 
additional  inspections  as  desired  to 
increase  their  confidence  in  the  parts 
quality. 


required  for  Class  S  or  substitute  Class 
S  parts  since  these  parts  receive  govern¬ 
ment  or  government  designated  source 
inspection.  The  procedures  for  performing 
supplier  surveillance  are  the  same  for 
subtier  contractors  as  for  the  prime. 
However,  the  prime  contractor  has  the 
additional  management  responsibility  to 
designate  representatives  as  necessary. 

Recommendations 

1.  Prime  contractors  implement  better 
flowdown  of  parts  requirements. 

a.  Define  parts  program  early 

b.  Assure  maximum  prime/sub  coopera¬ 
tion 

c.  Conduct  tasks  centrally  to  avoid 
duplication 

2.  Subcontractors  become  familiar  with  SD 
parts  documents. 


Supplier  Control  and  Surveillance 

Prime  Contractor  Subtier  Contractor 

Perform  surveillance  Perform  surveil- 

or  designate  repre-  lance  if  required 
sentative 

Verify  fabrication  to  Same 

manufacturing  baseline 
and  purchasing  document 
requirements 

Monitor  change  Same 

requests 

In-process,  final  Same 

inspections 

Qualification  require-  Same 
ments 


Notify  PMPCB  of 
problems 


Notify  prime  and 
PMPCB  of  problems 


Supplier  control  and  surveillance  is 
defined  as  the  monitoring  by  the  contract¬ 
or  of  his  designated  representative,  of 
the  manufacturing,  inspection,  and  test 
operations  of  a  supplier,  based  on  pre- 
established  criteria.  It  includes 
contractor  source  inspection.  Most  Space 
Division  programs  have  implemented  some 
type  of  supplier  surveillance.  It  is  not 


3.  Primes  and  subs  cooperate  in  coordina¬ 
ted  procurement  of  parts. 

4.  Primes  and  subs  purchase  highest  quality 
parts  available. 

5.  SD  document  required/recommended  flow- 
down  of  parts  requirements. 

6.  Contractors/users  send  recommended 
improvements  (in  writing)  to  SD/AQT. 

After  evaluating  the  implementation  of 
parts  control  programs  on  Space  Division 
contracts,  several  recommendations  are 
offered. 

An  important  lesson  learned  is  to  define 
the  parts  program  as  early  as  possible. 

Some  programs  have  defined  most  of  the 
parts  plan  details  by  the  contract  start 
date.  Centrally  conducted  tasks,  e.g., 
supplier  surveillance,  coordinated  procure¬ 
ment,  and  preparation  of  part  specifications 
will  reduce  duplication  of  effort. 

Becoming  familiar  with  the  requirements  of 
Space  Division  parts  documents  before  they 
are  on  contract  should  improve  proposals. 

As  Col  VanMeter  mentioned,  it  should 
prevent  offerors  from  accepting  efforts  for 
which  they  aren't  fully  prepared,  and  en¬ 
able  a  more  timely  implementation  of  the 
MIL  Spec  requirements. 


Repeating  the  advantages  of  coordinated 
procurement,  they  Include  reduced  parts 
documentation,  better  schedules,  less 
Inspection  effort,  and  lower  costs. 

Purchasing  the  highest  quality  parts  availa¬ 
ble  Is  essential  for  achieving  mission 
success. 

Documenting  recommendations  and  require¬ 
ments  of  the  better  methods  for  flowing 
down  the  piece  parts  requirements  has  been 
considered.  Also,  preparing  a  seminar  or 
training  on  the  specific  requirements  of 
MIL-STDs  1546  and  1547.  We  would  be  willing 
to  accept  an  action  Item  for  these  efforts. 

We  have  heard  criticisms  and  comments  on 
the  Space  Division  documents  and  would 
like  to  receive  some  sound  change  pro¬ 
posals  with  supporting  rationale  and  data. 


CONTRACTING  FOR  GOODS  AND  SERVICES 
A  QUALITY  PROBLEM  NOW  -  THE  FUTURE?? 
by 

Charles  R.  Mercer 
Director,  Quality  Assurance 
McDonnell  Douglas  Electronics  Company 


Good  afternoon  ladies  and  gentlemen.  I 
would  like  to  welcome  you  to  Session  2 
of  Workshop  C,  Subcontract  Management  - 
The  Prime  and  Subcontractor's  Viewpoint. 

I  would  like  to  make  a  few  opening  re¬ 
marks  relative  to  this  general  discus¬ 
sion  as  a  way  prefacing  our  activity 
this  afternoon.  I  will  try  to  make  this 
short  and  to  the  point  and  at  the  same 
time  put  forth  to  you  the  issues  as  I 
see  them  at  this  point  in  time. 

Why  is  this  an  appropriate  discussion? 
--Well,  in  my  opinion,  neither  the 
Government  nor  the  contractors  do  a  good 
job  with  their  contract  requirements 
flow  down.  Typical  problems  seen  in  to¬ 
day's  subcontracting  activity  involve 
overspecification  (most  typical).  This 
stems  from  a  lack  of  understanding  of 
the  true  requirements  and  a  ''CYA"  atti¬ 
tude  toward  taking  the  time  to  actually 
analyze  the  proper  flow  down.  Qn  the 
other  hand  however,  is  a  situation  where, 
in  fact,  proper  flow  down  of  all  the 
appropriate  requirements  is  not  accomp¬ 
lished.  Both  of  these  gross  errors  have 
a  tendency  to  introduce  down  stream  con¬ 
troversy  and  ultimately  delays  in  deliv¬ 
ery  of  conforming  qualified  hardware. 

When  you  add  delays  and  conflicts  to¬ 
gether  they  spell  only  one  thing  -  pro¬ 
ducts  cost  too  much. 

A  good  question  one  might  ask  is;  Who  is 
to  blame  for  this  situation?  Is  it  the 
Government?  Is  it  Industry?  Is  it  both? 
I  believe  the  answer  is  that  both  enti¬ 
ties  share  an  equal  responsibility  for 
this  situation. 

Where  did  we  go  wrong?  Typical  diffi¬ 
culties  are:  reliance  on  boiler  plate, 
not  enough  time  to  do  it  right  (typic¬ 
ally  based  on  unrealistic  schedule  re¬ 
quirements),  not  enough  technical  talent 
used  in  the  actual  contractual  require¬ 
ments  and,  unfortunately,  reliance  upon 
using  Mil-Specs  called  out  in  their  en¬ 
tirety  within  the  specification. 


What  is  the  impact  of  these  mistakes?  - 
-  But,  before  I  discuss  that,  I  feel  that 
all  of  you  can  somehow  identify  with  the 
points  I  raised  just  a  second  ago.  I 
know  of  no  contract  between  the  Govern¬ 
ment  and  the  prime  or  the  prime  and  his 
subcontractors  that  have  not  experienced, 
at  least  to  some  extent,  difficulties 
related  to  the  aforementioned  mistakes. 
Getting  back  to  the  impact  of  these  mis¬ 
takes;  the  following  are  prime  examples 
of  real  program  impacts.  1.  As  I  men¬ 
tioned  before,  products  and  services  cost 
more.  2.  The  delivery  schedules  are 
also  harder  to  meet.  This  type  of  prob¬ 
lem  is  typically  exacerbated  by  delays 
in  moving  products  through  manufacturing 
due  to  specmanshtp  by  the  contractors  or 
the  Government  QA  representative.  3. 
Last;  but  not  least,  as  we  contractors 
are  well  aware,  delay  in  payments  after 
the  work  is  done  is  often  caused  by  cer¬ 
tain  "procedure  difficulties".  At  this 
point,  a  little  history  review  is  in 
order. 

I  would  like  to  bring  a  contrast  to  your 
attention.  This  example  is  intended  to 
have  its  shock  value  and  it  is  not  neces¬ 
sarily  illustrative  of  what  I  think  to¬ 
day’s  contracts  should  look  like.  How¬ 
ever,  I  think  it  will  be  amusing  and  per¬ 
tinent  to  our  disucssions  here  today.  We 
should  consider  that  our  predecessors, 
although  considered  to  be  non-technically 
oriented,  may  have  had  some  good  ideas. 
The  example  is  a  three  page  Signal  Corps 
specification  generated  in  1907,  which  is 
purported  to  be  the  Wright  Brothers 
initial  contract.  As  I  quickly  flash 
these  slides  before  you,  I  would  like  to 
call  your  attention  to  the  sections  I 
have  marked;  specifically  on  the  first 
slide:  Inspection  --  The  Government 
reserves  the  right  to  inspect  any  and 
all  processes  of  manufacture.  Obviously 
no  special  requirements  were  levied  upon 
the  contractor  by  this  inspection  clause. 

On  Slide  number  2,  item  number  4,  is  an 
interesting  concept  that  has  been  kicked 
around  as  a  possible  contractual  incen¬ 
tive  at  numerous  symposia  in  the  past. 
Obviously,  in  1907  the  Government  rec¬ 
ognized  the  value  of  a  quality  incentive 
and  included  it  as  you  can  see  in  the 
specification.  The  last  item  that  I 
wish  to  call  your  attention  to  is  the 


general  requirements  for  how  the  machine 
should  perform  as  opposed  to  exactly 
how  it  should  be  done.  It  is  ob¬ 

vious  that  the  requirement  for  "suffic¬ 
ient"  simple  construction  and  operation 
requirements  such  that  an  intelligent 
man  could  become  proficient  in  its  use 
within  a  reasonable  length  of  time  is 
really  stating  the  requirements  in  gen¬ 
eral  terms. 

I 

To  further  illustrate  my  point  and  to 
bring  you  forward  in  time,  the  next 
slide  identifies  a  general  requirement 
type  specification  generated  to  the 
Boeing  Aircraft  Company  in  1936.  In 
this  specification  for  a  multi-engine 
bomber,  only  the  five  general  require¬ 
ments  noted  defined  what  the  Government 
had  in  mind.  The  results  of  these  re¬ 
quirements  and  the  contractors  ingenuity 
in  satisfying  them,  produced  a  rather 
significant  airplane.  I  am  speaking  of 
Boeing  Model  299,  alias  the  B-17.  It 
is  also  interesting  to  note  that  Model 
299  flew  eleven  months  after  the  con¬ 
tract  was  awarded. 

I  would  now  like  to  discuss  the  present 
situation,  as  I  see  it,  with  respect  to 
contracting  for  goods  and  services.  It 
is  obvious  that  the  Government  has  be¬ 
come  increasingly  aware  of  getting  more 
for  its'  buck.  One  way  of  doing  this 
is  to  streamline  the  contract  activity. 

I  feel  that  the  latest  developments  in 
this  area  are  interesting  to  review. 

I  will  not  attempt  to  read  these  examples 
to  you  from  the  slides;  however,  the 
fact  that  various  agencies  have  revised 
these  documents  to  detail  a  more  intelli¬ 
gent  way  of  doing  business,  is  what  I 
feel  is  significant.  The  first  one,  put 
out  by  the  Office  of  Management  and 
Budget  (0MB)  is  Circular  A-109.  DoD  has 
also  been  active  in  this  area  revising 
documentation  such  as  the  following: 

DoD  Directive  5000.1,  5000.37,  4120.21 
and  4155.1.  All  these  f^ocuments  have 
been  revised  to  streamline  the  way  DoD 
acquires  its  hardware  and  services. 

The  Air  Force  has  also  been  looking  into 
this  area.  I  would  like  to  call  your 
attention  to  the  "Quality  Horizon  Study" 
conducted  by  Col.  Bernie  Weiss.  I  have 
taken  the  liberty  of  using  excerpts  from 
his  speech  at  the  AIA/NSIA/QRAC  Confer¬ 


ence  in  Williamsburg,  Virginia  in  October 
of  1979.  This  slide  provides  you  with 
the  background,  but  I  would  like  to 
specifically  comment  on  the  findings  re¬ 
lated  to  contracts.  The  study  con¬ 
cluded  that  the  Air  Force  is  not  doing 
as  well  as  their  commercial  counterparts 
in  contracting  for  goods  and  services. 
This  may  indicate  a  change  to  more  com¬ 
mercial  like  contracting  activity  on  the 
part  of  the  Air  Force.  Only  time  vill 
tell.  The  second  finding  that  I  thought 
was  interesting  to  see  was  the  realiza¬ 
tion  that  Government  effort  is  light 
on  the  front  and  heavy  on  performance 
verification.  The  study  recommended 
that  the  Government  effort  be  modified 
so  as  to  put  equal  emphasis  on  front 
end  and  performance  verification.  These 
are  obvious  moves  which  could  make  the 
future  better;  but  will  it??  The  future 
is  hard  to  predict,  but  it  seems  to  me 
that  reasonable  direction  has  been  given 
from  on  high.  The  real  question  is: 

"Will  the  implementation  of  the  DoD 
directive  by  the  PCOs  and  CAOs,  really 
pick  up  this  thrust?" 

From  my  point  of  view,  the  onus  is  on 
the  Government  to  do  a  better  job,  but 
the  contractors  must  share  the  responsi¬ 
bility  to  make  it  happen.  Both  contrac¬ 
tors  and  Government  must  be  responsible 
in  their  contracting  activity.  This 
involves  taking  the  initiative  to  in¬ 
corporate  the  concept  of  specification 
tailoring  in  order  to  get  the  most  for 
the  buck.  Both  should  promote  and  insist 
upon  a  non-adversarial  role  by  involved 
parties.  And;  lastly,  the  KISS  principle 
should  prevail  -  "Keep  it  simple  stupid." 

In  summary,  we  have  made  a  lot  of  mis¬ 
takes  in  contracting  and  invoking  con¬ 
tractual  requirements.  We  should  have 
learned  something  from  these  mistakes. 

It  is  obvious  that  we  do  not  have  to  con¬ 
tinue  to  make  these  mistakes  ad  nauseum. 

I  believe  the  proper  groundwork  is  laid 
and  that  the  opportunity  is  available 
on  each  new  contract  to  use  common  sense 
in  contracting  activity.  Give  considera¬ 
tion  to  not  accepting  anything  less  in 
your  company  contracting  affairs  than  you 
would  in  your  own  personal  contracting 
affairs.  This  approach  will  meet  our 
mutual  goal  of  more  product  for  the  buck. 
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CONTRACTING  FOR  GOODS  AND  SERVICES 
A  QUALITY  PROBLEM  NOW  -  THE  FUTURE?? 


PRESENTED  BY 

CHARLES  R.  MERCER 


FOR 

1980  CONFERENCE  AND  WORKSHOPS  ON  MISSION  ASSURANCE 
29  APRIL  1980 


PAST 

0  WHY  IS  THIS  AN  APPROPRIATE  TOPIC?? 

0  NEITHER  GOVERNMENT  NOR  THE  CONTRACTORS  DO  A 
GOOD  JOB  WITH  CONTRACT  REQUIREMENTS  (CLAUSES) 

0  OVER  SPECIFY  (MOST  TYPICAL)  CYA 
0  UNDER  SPECIFY 

0  RESULT  -  THINGS  COST  TOO  MUCH 
0  WHO  IS  TO  BLAME  FOR  THIS  SITUATION? 

0  GOVERNMENT 
0  INDUSTRY 


0  WHERE  no  WE  GO  WRONG? 

0  BOILER  PLATE 

0  NOT  ENOUGH  TIME  TO  DO  IT  RIGHT  -  UNREALISTIC 
SCHEDULES 

0  NOT  ENOUGH  TECHNICAL  TALENT  USED  IN  CREATION 
OF  THE  CONTRACTUAL  REQUIREMENTS 

0  RELIANCE  UPON  MIL-SPECS  (IN  THEIR  ENTIRETY) 

TO  SPECIFY  REQUIREMENTS 

0  WHAT  IS  THE  IMPACT  OF  THESE  STANDARD  MISTAKES? 

0  PRODUCTS  AND  SERVICES  COST  MORE 

0  DELIVERY  SCHEDULES  ARE  HARD  TO  MEET 

0  DELAYS  IN  MOVING  PRODUCTS  THROUGH  MANUFACTURING 
DUE  TO  SPECSMANSHIP  BY  THE  CONTRACTORS  QA  OR  THE 
GOVERNMENT  QA  REPRESENTATIVE 

0  DELAYS  IN  PAYMENT  AETER  WORK  IS  DONE  CAUSED  BY 
'PROCEDURAL  DIEFICULTIES" 


1936:  AAF  SPEC  FOR  A  MULTI-ENGINE  BOMBER 
0  BOMB  LOAD 
0  CEILING 
0  SPEED/RANGE 

0  RUNWAY  LENGTH  FOR  TAKEOFF 
0  CLIMB  RATE 

RESULT:  BOEING  MODEL  299  ALIAS  THE  3-17 


Note:  First  model  299  flew  in  eleven  months  after 


CONTRACT  AWARD. 


PRESENT 

0  THE  GOVERNMENT  HAS  BECOME  INCREASINGLY  AWARE  OF  THE 
PROBLEM  OF  GETTING  MORE  FOR  ITS  BUCK.  ONE  WAY  OF 
DOING  THIS  IS  TO  STREAMLINE  ITS  CONTRACTING  ACTIVITY. 
THE  LATEST  DEVELOPMENTS  ARE  INTERESTING  TO  REVIEW. 

0  THE  RE-THINKING  STARTS  AT  THE  HIGHEST  LEVELS. 

OFFICE  OF  MAN.AGEMENT  AND  BUDGETS  (0MB)  CIRCULAR  A-109 
General  Policy: 

THE  POLICIES  OF  THIS  CIRCULAR  ARE  DESIGNED  TO  ASSURE 
THE  EFFECTIVENESS  AND  EFFICIENCY  OF  THE  PROCESS  OF 
ACQUI.RING  MAJOR  SYSTEM.S.  THEY  ARE  BASED  ON  THE  GENERAL 
POLICY  THAT  FEDERAL  AGENCIES,  WHEN  ACQUIRING  MAJOR 
SYSTEMS  WILL: 

(a)  express  needs  and  PROGRAM  OBJECTIVES  IN  MISSION 
TERMS  AND  NOT  EQUIPMENT  TERMS  TO  ENCOURAGE  IN¬ 
NOVATION  AND  COMPETITION  IN  CREATING,  EXPLORING, 
AND  DEVELOPING  ALTERNATIVE  DESIGN  CONCEPTS. 

(BJ  PLACE  EMPHASIS  ON  THF  INITIAL  ACTIVITIES  OF  THE 
SYSTEM  ACQUISITION  PROCESS  TO  ALLOW  COMPETITIVE 
EXPLORATION  OF  ALTERNATIVE  SYSTEM  DESIGN  CONCEPTS 
IN  RESPONSE  TO  MISSION  TERMS. _ 

0  ’DOD  DIRECTIVE  5000.1  -  MAJOR  SYSTEM  ACQUISITION 

0  STRESSES  FLEXIBILITY 
0  REQUIRES  GREATER  DEPENDENCE  UPON  MISSION 
REQUIREMENTS 

0  DOD  DIRECTIVE  5000.37  -  ACQUISITION  AND  DISTRIBUTION 
OF  COMMERCIAL  PRODUCTS 
0  ACQUIRE  OFF-THE-SHELF  PRODUCTS 
0  ELIMINATE  UNNECESSARY  GOVERNMENT  SPECIFICATIONS 
FOR  COMMERCIAL  PRODUCTS 

0  OPTIMIZE  THE  GOVERNMENT  ADVANTAGE  WHILE  MINIMIZING 
THE  ADMINISTRATIVE  BURDEN 

0  'DOD  DIRECTIVE  R120,21  -  APPLICATION  OF  SPECIFICATIONS, 
STANDARDS,  AND  RELATED  DOCUMENTS  IN  THE  ACQUISITION 
PROCESS 

0  AVOID  BLANKET  IMPOSITION  OF  SPECS,  STANDARDS 
AND  RELATED  DOCUMENTS.  TAILOR  TO  MEET  PROGRAM 
REQUIREMENTS. 

0  CONTRACTUAL  IMPOSITION  SHALL  BE  LIMITED  TO  THOSE 
DOCUMENTS  SPECIFICALLY  CITED  IN  THE  CONTRACT 
0  ALL  OTHER  DOCUMENTS  INCORPORATED  THRU  REFERENCE 
WITHIN  CITED  DOCUMENTS  SHALL  BE  CONSIDERED  AS 
'GUIDANCE'  OR  "INFORMATION'. 


0  DOD  DIRECTIVE  <1155.1  -  QUALITY  PROGRAM 

0  EFFECTIVELY  ASSURE  THAT  ONLY  MINIMUM  ESSENTIAL 
QUALITY  AND  RELATED  REQUIREMENTS  ABE  SPECIFIED 

0  TAILOR  CONTRACTUAL  QUALITY  REQUIREMENTS  TO  MEET 
THE  NEEDS  OF  EACH  ACQUISITION 

0  MILITARY  HANDBOOK  -  PROPOSED  -  D0D-HDBK-2<15A 
SUPERCEDING  MIL-STD-2<(8(AS)  1  APRIL  1977 

0  QUALITY .HORIZON  STUDY  -  USAF,  COL.  'BERNIE'  WEISS  - 
EXCERPTS  TAKEN  FROM  HIS  SPEECH  AT  AIA/NSIA/QRAC 
CONFERENCE  IN  WILLIAMSBURG,  VA.,  OCTOBER  1979 

0  BACKGROUND  -  6  MONTHS  STUDY 

0  COVERED  COUNTRIES  AROUND  THE  WORLD 
0  66  COMPANIES  AND  GOVERNMENT  AGE’ DIES  vfSITED 
0  MAJOR  FINDINGS  AS  RELATED  TO  CONTRACTING 
0  AF  NOT  DOING  AS  WELL  AS  THEIR  COMMERCIAL 
COUNTERPARTS  IN  CONTRACTING  FOR  GOODS  AND 
SERVICES 

0  GOVERNMENT  EFFORT  IS  LIGHT  ON  IHE  FRONT  END 
AND  HEAVY  ON  PERFORMANCE  VERIFICATIOT -  SHOULD 
BE  A  BALANCED  EFFORT 

THESE  ARE  OBVIOUSLY  POSITIVE  MOVES  WHICH  SHOULD  MAKE  THE 
FUTURE  BETTER.  BUT  HILL  IT?? 


FUTURE 

0  REASONABLE  DIRECTION  HAS  BEEN  GIVEN  FROM  ON-HIGH, 
BUT  HILL  THE  IMPLEMENTERS  OF  DOD  DIRECTIVES,  THE 
PCOs  AND  CAOs,  REALLY  JUMP  ON  THE  BAND  WAGON? 

0  FROM  MY  POINT  OF  VIEW,  THE  ONUS  IS  UPON  THE 
GOVERNMENT  TO  DO  A  BETTER  JOB,  BUT  THE  CONTRACTORS 
MUST  SHARE  THE  RESPONSIBILITY  TO  "MAKE  IT  HAPPEN'. 

0  CONTRACTORS  AMD  GOVERNMENT  MUST  BE  RESPONSIBLE 
0  REVIEW  CONTRACTS  WITH  THE  IDEA  OF  TAILORING 
TO  GET  THE  HOST  FOR  THE  BI'CK 
0  NOT  TAKE  THE  EASY  HAY  OUT  AND  ENVOKE  COMPLETE 
SPECS 

0  COOPERATE  ON  A  NON-ADVERSARIAL  BASIS  T"  REACH 
THE  GOAL  OF  AN  OPTIMIZED  CONTRACT 
0  THE  KISS  PRINCIPLE  SHOULD  PREVAIL  -  'KEEP  IT 
SIMPLE  STUPID'. 


In  Coordination  with  Industry 


SUWARY: 

0  WE  HAVE  MADE  A  LOT  OF  MISTAKES  IN  CONTRACTING 
AND  INVOKING  CONTRACTUAL  REQUIREMENTS 
0  WE  DON'T  HAVE  TO  CONTINUE  TO  MAKE  THE  SAME 
MISTAKES  AD  NAUSEUM 
0  GROUNDWORK  IS  LAID 

0  OPPORTUNITY  IS  AVAILABLE  ON  EACH  NEW  CONTRACT 
0  UTILIZE  THE  TOOLS  -  COMMON  SENSE  -  AVAILABLE 
0  THE  PEOPLE  WHO  MAKE  AND  ENFORCE  THE  CONTRACTS 
ARE  THE  KEY  TO  THIS  EFFORT.  DON'T  ACCEPT  IN 
YOUR  CONTRACT  EFFORT  ANYTHING  LESS  THAN  YOU 
WOULD  IN  YOUR  PERSONAL  CONTRACT  AFFAIRS.  THIS 
APPROACH  HILL  MEET  OUR  MUTUAL  GOAL  OF  MORE 
PRODUCT  FOR  THE  BUCK. 


Flow-Down  of  Quality  Requirement? 

TO  Semiconductor/Microelectronic  Suppliers 

Conrad  H.  Zierdt^  Jr. 

Bell  Telephone  Laboratories 
Allentown^  Pa, 

Chairman^  JEDEf.  JC-13,  Committee 
on  Government  Liaison 


Old,  Large,  Broad-Product-Line  Company, 

Data  from  2/3  of  total  plants  available  for 
SURVEY;  year-to-date  Oct.  1977 : 

Total  UQ  audits  and  surveys. 

1300  COMPANY  MAN-HOURS  EXPENDED. 

1.5-2  VISITOR  MAN-DAYS  (IN-PLANT)  PER  VISIT 

Comment:  40Z  of  visits  superficial  or 
superfluous 


By  What  Authority? 


DESC  -  MIL-S-19500  Appendix  D 
MIL-M-38510  Appendix  A 

These  are  detailed  and  specific  for 
discrete  devices  and  microcircuits, 
respectively. 

DCASR  -  and  Customers  MIL-Q-9858A 

Para.  1.3  "The  t quality!  program  shall 

INCLUDE  AN  EFFECTIVE  CONTROL  OF  PURCHASED 
materials  and  SUBCONTRACTED  WORK." 

Section  5  "The  effectiveness  and  integrity 
OF  the  control  of  quality  by  his  suppliers 
shall  be  assessed  and  REVIEWED  BY  THE 
CONTRACTOR  AT  INTERVALS  CONSISTENT  WITH  THE 
COMPLEXITY  AND  QUANTITY  OF  PRODUCT." 


Quality 


r  Systems  / 
)  Program  \ 
I  Control  ( 
V^Assuranc^ 


Surveys 


Parkinson's  Law  in  Action 

VIA 

MIL-Q-9858 

AND  The  Establishment 


Old,  Medium,  Fairly-Broad  Product  Line  Company 

Average  for  past  2  years,  one  location: 

17  VISITS  per  year 
$10,000/yr.  company  expenditure 
2  VISITORS,  2.  days  per  VISIT 

Comment:  Visits  begin  October  and  taper  off 
IN  April  at  California  location. 


Old,  Large,  Broad-Product-Line  Company 

Visits  by  prime/sub-contractors  under 
MIL-0-9858A  only: 

mi  B.  IS  m  77  YTD 
21  11  8  12  13 

5-6  COMPANY  MAN-HOURS  PER  VISIT 
Z  VISITORS,  1  DAY  TYPICAL. 

Comment;  Both  ebe.  (when  bidding)  and 
POST  (after  contract  award) 
visits  seem  unnecessary. 
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’.Ihat's  The  Problem? 


New^  Small^  Restricted-Product-Line  Company 

Data  for  first  nine  months  of  1977 

67  visits 
127  VISITORS 

1356  COMPANY  MAN-HOURS  DURING  VISITS 
(no  figure  for  preparation  time) 

Comments:  1)  Probably  fewer  visits  if  not 
LOCATED  IN  SILICON  VaLLEY. 

2)  Visitors  include  most  aero¬ 
space  CONTRACTORS. 


Too  MANY  visits: 

Eat  supplier  quality-assurance  people's  time 
Divert  manufacturing  work  force 
Cause  unnecessary  cost  to  the  Government: 
Directly,  for  travel. 

Indirectly,  as  cost  to  device  suppliers 

Too  many  Experts: 

Convey  too  many  prescriptions  for  instant 
reliability. 

Ask  for  far  more  than  specifications  require. 


Parkinson  Prevails 

Rate  of  visitation  has  not  been  noticeably  reduced, 
SINCE  1965  OR  1978. 


WHO  SURVEYS  ? 

DESC-EQ  -  1  VISIT  PER  YEAR  PER  PRODUCT  LINE 
Full-Time  engineering  personnel 


Implications: 

The  activity  continues  at  a  level  consistent  with  the 

MONEY  AVAILABLE  AND  THE  ESTABLISHED  FUNCTIONAL  CROUPS/ 
PATTERNS  IN  EQUIPMENT  CONTRACTORS'  HOUSES. 

CASE  IS  FOR  God,  motherhood,  and  ap^le  pie,  but  not 

ACTION. 


DCASR  -  Sporadic 

Qualifications  vary  widely. 


Customer  Reps.  -  Generally  1  visit  per  year 
Qualifications  vary  widely. 


Recommendations 

1)  Change  H1L-9-9853by  cutting  out  the  flow- 
down  (to  part  level)  provisions,  when  parts 
specifications  include  appropriate  requirements, 

2)  Stop  paying  for  equipment-contractor  surveys, 

UNDER  THE  SAME  CIRCUMSTANCES. 


Common  Beliefs/Postures: 

Known  quality  factors  better  than 
device  manufacturer. 

Are  better  motivated. 


Summary  of  Statistics 

Four  (A)  visits  would  have  been  as  effective 
AS  WERE  TWO  HUNDRED  SEVEN  (207)  IN  3/A  OF  1977, 

Conservatively,  for  203  excess  surveys: 

Travel  BSSOO/trip  101,500 

Contractor  time  a  2  man-days/trip.  126,875 

Supplier  time  a  A  man-days/trip  253,750 

Avoidable  cost  -  $A82,125 


What's  Being  Done? 

19/8  Memo  -  Lester  Fox,  Director,  DMSSO  to  Hr.  John  Hittino,  Deputy 
Director^  OUSD/RsE 

"Prime  contractors  are  not  making  epfective  use  of  the  suppliers' 

OBJECTIVE  EVIDENCE  OF  QUALITY  AND  THE  CONTRACTORS'  OWN  RECEIVING 
inspection  TO  REDUCE  THE  NUMBER  DF  ON-SITE  AUDITS.  FURTHER  THERE 
IS  NO  "reciprocity'  among  the  Services  and  primes  in  audits  already 
CONDUCTED  where  A  DETERMINATION  OF  SUITABILITY  HAS  BEEN  MADE." 

19/8  Memo  -  Capt,  C.  H,  Lamb,  DLA-QEL  to  Staff  Director  for  OA, 

OUSD  R*E  (SOS) 

"The  resultant  situation  is  a  steady  stream  of  visitors  to  the  more 
important  componen.'  suppliers. 

A  FIRST  STEP  IN  THE  AILEVIFTION  OF  THIS  SITUATION  WOULD  BE  TO  DIRECT 
THE  specification  PREPARING  ACTIVE'  TO  CONFIDER  CM.'.SGF  T'l  VHi 

pertinent  specification  paragraphs.  The  change  should  not  afflu.- 
ike  basic  philosophy  of  KlL-n-9358A.  h  should  encourage  joint 
surveys  by  industry  and  reciprocity  relative  to  acceptable  audit  or 

SURVEY  RESULTS." 

1969  -  65  -  Correspondence  between  same  Government  functions  and 
EIA,  IN  IDENTICAL  vein. 

CASE  ■  Coordinating  Agency  for  Supplier  Evaluation 

An  association  of  aerospace  and  nuclear  industry  companies  - 
furnishes(eo)  a  register  of  quality-control  evaluated  suppliers. 
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PUBLIC  LAW  95-507 
by 

Sewell  T.  Kauffman,  Manager  - 
Small  Business  Utilization 
McDonnell  Douglas  Corporation 
St.  Louis,  Missouri 


I  was  asked  to  discuss  Public  Law  95-507 
and  its  impact  upon  contractors--but  I'm 
sure  that  I'm  going  to  learn  far  more 
from  you  than  you  learn  from  me. 

Any  procurement  personnel  of  any  large 
company  negotiating  a  contract  with 
the  Federal  Government  exceeding 
$500,000  (or  $1,000,000  for  construction), 
since  the  first  of  the  year,  will  have 
gotten  up  to  his  eyeballs  in  corres¬ 
pondence  concerning  95-507.  Dale 
Church,  Deputy  Under  Secretary  of 
Defense,  said  that  there  is  "absolute 
chaos"  in  the  implementation,  with  goals 
and  means  of  implementation  which  are 
turning  out  to  be  "meaningless." 

Its  not  that  the  law  itself  is  so 
complicated.  It's  the  fact  that  every 
Federal  agency  has  made  its  own  inter¬ 
pretation  of  the  law  and  expanded  on 
its  requirements.  To  top  this  off, 
the  Office  of  Federal  Procurement 
Policy  has  dragged  its  feet  in  issuing 
its  implementing  instructions.  Therefore, 
nobody  is  in  a  position  to  tell  the 
many  procurement  contracting  officers 
that  they  are  asking  for  too  much. 

Let's  look  at  the  Law  and  see  what  is 
required--just  six  points: 

1.  Percentage  Goals; 

2.  The  name  of  the  Subcontract 
Administrator  and  his  duties. 

3.  A  description  of  the  efforts 
we'll  take  to  assure  that 
Small  &  Disadvantaged  firms 
have  an  equitable  opportunity 
to  compete. 

4.  Assurances  we'll  flow-down  the 
clause. 

5.  Assurances  we'll  submit 
periodic  reports  and  cooperate 
in  any  studies;  and 

6.  A  recitation  of  the  types  of 
records  we'll  keep. 

Now  look  at  what's  required: 

1.  Percentage  Goals  -  Who's 

kidding.  The  Air  Force  wants, 
in  addition  to  percentages. 


the  total  dollar  value  of  planned  sub¬ 
contracting,  total  to  small,  and  total 
to  disadvantaged  firms.  They  want  a  des¬ 
cription  of  the  principal  product  and 
service  areas  to  be  subcontracted  and  an 
identification  of  those  areas  where 
small  and  disadvantaged  firms  will  be 
used.  The  Air  Force  wants  the  con¬ 
tracting  officer  to  be  furnished  the 
names  and  locations  of  the  principal 
small  and  disadvantaged  subcontractors 
including  the  type  of  product  or 
service  and  dollar  value  thereof  to 
be  awarded  to  each  principal  subcon¬ 
tractor.  The  Air  Force  wants  the 
method  used  in  developing  the  proposed 
subcontracting  goals.  For  example--Did 
we  use  company  source  lists,  PASS, 

NMPC,  etc.  And  lastly,  they  want  to 
know  the  method  used  in  determining 
indirect  and  overhead  costs  to  be 
allocated  to  that  particular  acquisition. 

This  is  all  included  under  the  Public 
Law  95-507  innocent  heading  titled 
"Percentage  Goal’s." 

2.  Name  of  the  individual--This  is  not 
too  bad.  The  Air  Force  suggests  that 
the  individual  be  placed  in  the 
organ' zational  structure  for  reporting 
directly  to  the  chief  executive  or  vice 
president. 

3.  A  description  of  the  efforts  to 
assure  small  and  disadvantaged  firms' 
participation--In  this  we  must  include 
company-wide  policy  statements,  proof 
of  management  interest  and  involvement, 
"feedback"  briefings,  corporate  and 
divisional  goals,  etc.  We  must  establish 
and  describe  personnel  training  and 
motivation  programs.  We  must  describe 
our  special  assistance  program  to  help 
small  and  disadvantaged  firms.  How  are 
they  considered  in  our  Make-or-Buy  plans. 
And  we  must  describe  our  efforts  to 
participate  in  counselling  activities 
sponsored  by  business  and  government 
groups . 

4.  Clause  Flow  Down--Not  too  bad.  We 
have  to  acknowledge  that  we  have  the 
obligation  to  flow-down  the  clause. 
However,  this  turns  out  to  be  a  real 
sleeper.  The  Navy  and  OFPP  stipulate 
that  the  Prime  Contractor  is  responsible 
for  monitoring  compliance  by  his  sub¬ 
contractors.  The  NAVPRO's  responsi- 


bility  in  this  area  is  to  see  that  the 
Prime  is  taking  action  to  see  that  the 
subcontractor  is  complying  with  the 
provisions  of  the  subcontracting  plan. 
OFPP  believes  that  Prime  Contractors 
should  not  only  flow  down,  monitor 
and  obtain  statistical  data  from  sub¬ 
contractors,  but  also  must  include  in 
their  plans  the  management  techniques 
that  will  be  employed 'to  accomplish  this 
task.  I  might  add  that-  in  1979  McDonnell 
Douglas  had  699  purchase  orders,  not 
to  mention  the  innumerable  modifications 
and  change  orders  involved,  which 
exceeded  $500,000. 

5.  Reports  and  studies.  On  the  surface 
this  was  left  pretty  much  alone.  We 
have  to  include  in  the  Plan  assurance 
that  we  will  make  the  required  reports 
and  cooperate  in  any  studies.  Much 
hinges  on  the  issuance  of  OFPP's 
implementing  instruction  regarding 
reports.  They  have  proposed  the  use  of 
a  Federal  Quarterly  Individual  Sub¬ 
contracting  Report  as  well  as  a  Federal 
Quarterly  Summary  Subcontracting  Report. 
The  data  required  includes  awards  to 
Small  and  Large  Businesses,  Disad¬ 
vantaged  Businesses,  Women-Owned 
Businesses,  Labor  Surplus  Area  Awards, 
Awards  to  Non-Profit  and  Foreign  Sub¬ 
contractors;  number  of  active  contracts 
having  small  business  goals,  number  with 
work  completed,  number  completed  which 
met  small  business  goals,  etc. 

6.  And  last  but  not  least,  we  come  to 
Records— The  Public  Law  wants  a 
recitation  of  the  types,  the  estab¬ 
lishment  of  source  lists  for  small  and 
disadvantaged,  and  a  means  to  identify 
our  efforts  to  award  subcontracts  to 
such  small  business  concerns. 

You  now  have  a  fairly  good  picture  of 
the  original  Public  Law  requirements 
and  what  has  happened  in  the  imple¬ 
mentation  of  this  law.  For  the  first 
two  months  of  this  year,  our  subcon¬ 
tracting  plans  were  short,  concise  and 
easily  understood.  Or,  so  we  thought. 

For  the  first  two  months  not  a  one  was 
approved.  We  then  began  to  wax  elequent, 
became  more  descriptive  and  verbose.  The 
bigger  the  plan,  the  greater  chance  we 
had  of  its  acceptance.  We  have  now 
reached  that  level  of  understanding  with 
our  usual  customers  wherein  our 


communications  are  good.  Our  plans  are 
being  accepted  with  few  exceptions.  The 
administration  of  95-507  is  causing  us 
to  add  an  estimated  ten  persons  to  our 
various  MDC  small  business  offices.  And 
we  still  don't  know  how  many  more  will 
be  required  when  we  get  into  the 
tracking  of  all  these  plans,  coordinating 
studies  and  making  reports.  I  have  not 
included  the  requirements  imposed  on 
our  Contracts  Department  where  each 
plan  has  to  be  negotiated  with  the 
customer.  Since  January  of  this  year, 
McDonnell  Aircraft  Company  has  submitted 
over  50  subcontracting  plans.  In  the 
few  cases  wherein  we  stated  no  plan  was 
required  because  no  subcontracting 
possibilities  existed,  we  merely  started 
a  war  of  words  because  the  PCO  felt  he 
should  have  a  plan. 

In  spite  of  all  the  problems  involved  in 
implementing  this  law,  we  are  reaching 
an  understandable  and  workable  relation¬ 
ship  with  the  PCO's.  You  can  compare 
this  with  a  shotgun  wedding.  It's 
starting  to  develop  into  a  marriage 
of  convenience--and,  who  knows,  some 
day  it  might  become  a  love  match. 
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OVER-THE-SHOULDER  AUDITS 


Dave  Arnott 

SENIOR  QUALITY  CONTROL  ENGINEER 
GENERAL  DYNAMICS  CONVAIR 

Quality  Assurance  Auditing  is  not  new  but 
prior  to  the  1960's  it  was  seldom  employed. 
During  the  60' s,  Convair,  because  of  their 
Quality  reputation  on  the  Atlas  missile,  was 
put  under  contract  to  provide  an  audit  pro¬ 
gram  to  indemnify  the  Defense  Department 
against  poor  process  and  quality  control. 

The  program  reported  directly  to  Robert 
MacNamara,  Secretary  of  Defense.  During 
the  five  years  of  its  existence,  with  a  crew  of 
eight,  it  proved  to  be  highly  rewarding  to  the 
Defense  Department. 

Government  agencies  analyzed  the  results  and 
concluded  that  economies  were  to  be  had 
without  undue  risk.  Accordingly,  they  re¬ 
duced  their  hardware  inspection  activity  in 
favor  of  comprehensive  audits. 

Through  the  Defense  Department  Audit  Pro¬ 
gram,  Convair  became  the  grandfather  of  the 
audit  system.  It  was  the  result  of  the  NASA 
Gaberiel-Dey  Audit  in  1971  of  the  Atlas  Cen¬ 
taur  program  that  Convair  was  funded  for  a 
full  scale  quality  audit  program.  This  was  a 
joint  effort  by  NASA  and  SAMSO. 

The  purpose  of  an  audit  program  is  to  verify 
that  the  Management  Procedures,  the  Product 
Design  Requirements  and  the  Production 
processes  are  being  conducted  to  the  require¬ 
ments,  The  audit  provides  visibility  to 
management  and  ascertains  that  the  program 
plan  adequately  implements  the  customer 
requirements.  The  placement  of  the  audit 
function  within  the  organization  is  very  im¬ 
portant.  It  must  report  on  a  staff  level  in 
order  not  to  be  influenced  by  the  functions 
being  audited.  Convalr's  Quality  Assurance 
Director,  our  highest  authority  in  Quality, 
reports  to  the  General  Manager.  The  Quality 
Assurance  Audit  Group  reports  directly  to  him. 


This  assures  the  Auditors  can  carry  out  the 
audit  objectives  without  interference  from  sub¬ 
tler  supervision.  What  are  these  objectives? 
Foremost,  of  course,  is  to  insure  that  the 
customer's  requirements  are  fully  met.  The 
first  customer  requirement  is  to  have  the 
contractor  conduct  self  audits  to  verify  his 
operations  and  prevent  the  production  of  non- 
conforming  supplies.  The  audit  is  also 
beneficial  to  management  by  minimizing  sur¬ 
prises  and  establishing  visibility  into  how 
well  the  total  operation  is  performing.  It 
further  establishes  customer  confidence, 
minimizing  the  necessity  for  the  customer, 
to  ccnduct  audits  which. are  time  consuming 
and  costly  to  the  customer  and  to  the  con¬ 
tractor.  There  is  a  further  objective  of 
determining  that  the  contractor  is  complying 
with  contractual  requirements  and  has  ade¬ 
quate  procedures  or  work  Instructions  to 
enable  conformance.  An  audit  also  evaluates 
the  product  through  spot  reinspection  to 
measure  the  results  of  the  entire  Quality 
System.  Last  but  not  least,  the  effectiveness 
of  the  corrective  action  system  must  be 
assessed  to  determine  its  ability  to  prevent 
the  same  discrepancies  from  happening 
twice.  We  do  this  by  conducting  specific 
types  of  audits  both  in  our  house  and  at  our 
suppliers. 

Audits  can  only  be  conducted  in  an  effective 
manner  when  the  auditor  is  provided  adequate 
time  to  prepare  a  comprehensive  audit  plan. 

He  must  expend  considerable  time  in  deter¬ 
mining  the  requirements  of  the  "Contract"  — 
what  does  the  contract  say?  He  must  study  in 
detail  the  engineering  requirements  ano  the 
specifications  Imposed  therein.  Further 
effort  must  be  expended  on  a  thorough  analysis 
of  the  contractor's  internal  procedures  and 
quality  disciplines  to  see  that  all  require¬ 
ments  are  compatible.  Work  Instructions 
must  also  be  reviewed  to  determine  if  they 
convey  the  information,  in  proper  sequence 
to  enable  the  worker  to  carry  out  the  job. 

Since  the  contractor's  Internal  auditors  are 
familiar  with  the  contractor's  procedures 
and  organization,  this  task  can  be  done  much 
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more  efficiently  than  It  could  be  accomplish¬ 
ed  by  customer  auditors  unfamiliar  with  the 
contractor's  systems  and  organization. 
Therefore,  the  Over-The-Shoulder  Audit  con¬ 
cept  becomes  a  very  cost  effective  method 
for  the  customer. 

Product  reinspection  audits  tell  us  how  we  are 
doing.  It  also  Is  a  measure  of  Individual 
Inspectors.  Keeps  them  on  their  toes  and 
enables  supervision  to  correct  and  relnstruct 
where  necessary.  At  the  time  of  relnspectlon 
audits,  planning  Is  reviewed  as  well  as  the 
engineering  drawing.  We  see  that  all 
engineering  requirements  are  reflected  In  the 
planning  aixi  that  engineering  la  complete  as 
to  calling  out  the  contractually  Imposed  speci¬ 
fications.  A  review  Is  made  of  previous  dis¬ 
crepancies  to  evaluate  the  effectiveness  of 
corrective  actions.  If  a  vendor  part  Is  in¬ 
volved,  a  review  of  receiving  Inspection  docu¬ 
mentation  is  made  to  determine  that  we  have 
the  proper  certifications  and  have  completed 
quality  engineering  requirements. 

Processing  Is  another  type  of  audit.  We,  like 
other  organizations,  have  experts  on  process¬ 
ing.  Documents  are  written  that  impose 
specific  disciplines  on  the  way  things  are 
processed.  We  must  assure  that  the  recipe 
Is  followed  in  detail  to  guarantee  that  the 
product  Is  right.  Process  certification  is  an 
Important  Ingredient.  Unless  you  know  that 
the  equipment,  solutions,  temperature,  work¬ 
ing  zones  of  a  heat  treat  oven  are  capable  of 
meeting  specification  requirements,  you  are 
lost.  You  must  also  know  that  the  people 
operating  this  equipment  have  a  thorough 
understanding  of  the  disciplines  imposed.  The 
auditor,  therefore,  checks  process  certifi¬ 
cation  and  personnel  certification.  The  recipe 
must  be  chedced  to  determine,  for  instance, 
that  the  chromium  plating  Is  not  accomplished 
prior  to  magnetic  Inspection.  This  situation 
would  nullify  the  Inspection  check.  On  top  of 
all  this,  the  auditor  keeps  his  eye  peeled  for 
safety  items  for  safety  Is  everyone's  business. 

Another  phase  of  auditing  Is  the  policing  of 
required  tests.  Here  we  must  assure  the 


company  that  the  test  procedure  incorpoirates 
all  those  things  that  were  determined  to  be  of 
value  by  Engineering.  We  must  also  concern 
ourselves  with  the  people  performing  the  test. 
We  do  not  want  the  type  of  employee  who  con¬ 
forms  to  the  trite  statement  that  "yesterday 
I  could  not  spell  engineer,  but  today  I  are 
one."  The  auditor,  therefore,  checks  to  see 
that  the  personnel  are  certified  by  Quality 
Engineering.  It  Is  obvious  that  if  we  have  a 
specific  pressure  requirement  we  would  want 
to  know  that  the  pressure  gage  being  used 
reflects  the  exact  pressure.  To  this  end, 
Calibration  becomes  a  most  important  Ingre¬ 
dient  and  must  be  observed  to  assure  ac¬ 
curacy.  The  auditors  are  only  in  the  area 
periodically  and  at  best,  can  only  sample  the 
performance.  The  back  bone  of  testing  is 
the  inspector  on  the  Jcb.  The  auditor  is, 
fiierefore,  very  concerned  that  inspection  Is 
following  the  test  and  that  the  inspector  knows 
his  business.  Documentation  of  test  results 
is  most  Important.  Should  something  go 
wrong  with  the  item  under  test,  it  Is  necessar> 
to  know  the  complete  details  of  the  test. 

Often  we  find  such  Important  things  as  time, 
temperature  and  pressure  have  been  omitted 
on  the  data  sheets.  We  check  for  this. 

Again,  audit  must  be  sensitive  to  all  safety 
requirements. 

A  satisfactory  cake  cannot  be  baked  without 
a  good  recipe.  In  our  business,  work 
Instructions  (planning)  Is  the  recipe.  Plan¬ 
ning  must  include  the  drawings  and  specifi¬ 
cations  requirements,  procedures  to  be  used, 
the  quality  requirements  to  be  met  and  the 
step-by-step  approach  to  each  event.  Our 
auditors  on  a  random  basis  audit  a  number 
of  pages  of  planning  to  determine  if  all  of  the 
disciplines  are  being  met. 

All  Quality  programs  must  have  a  road  map. 
This  Is  called  the  Quality  Program  Plan. 
Because  of  the  large  volume  of  paperwork, 
there  Is  often  a  hole  In  the  basket.  It  Is  the 
responsibility  of  the  auditor  to  Identify  these 
holes.  Therefore,  the  program  plan  Is 
audited  to  determine  If  the  plan  adequately 
implements  the  procedures  necessary  to 


produce  a  quality  product  In  compliance  with 
design  requirements.  Once  this  is  establish- 
ed,  the  audit  must  concern  itself  with  the  con¬ 
formance  to  procedures.  To  this  end,  a  com¬ 
prehensive  audit  is  conducted  of  systems  and 
procedures.  In  our  company  this  is  an  audit 
of  our  Division  Standard  Practices  and  their 
integration  with  the  various  manufacturing 
procedures.  This  is  to  assure  that  program 
plans,  manufacturing  instructions  and  quality, 
directives  are  in  synchronization  resulting  in 
a  uniform  system. 

Often  times  the  audit  function  is  called  upon 
for  special  audits.  These  may  be  requested 
by  managene  nt  or  the  customer.  A  case  in 
point  was  an  unexplained  rise  in  the  rejection 
rate  of  Convalr's  machine  shop.  Both  the 
customer  and  management  commissioned  the 
audit  function  to  go  in  and  try  to  determine  the 
causes.  Problems  were  identified  as  lack  of 
machine  capability,  poor  communication  be¬ 
tween  worker  and  supervision,  lax  inspection 
techniques,  tooling  problems  and  shop  lay  out. 
There  was  a  significant  pay  off  for  a  two  week 
effort  by  one  auditor.  We  have  initiated 
start  centers,  re-oriented  inspection,  remov¬ 
ed  paper  loads  from  shop  foreman  and  vastly 
improved  communication.  This  is  called 
corrective  action  -  it  pays  off  in  reduced 
rejections  and  Increased  efficiency. 

supplier  audits,  while  generally  the  same  as 
internal  audits,  do  have  significant  differences. 
Of  prime  import  is  the  communication  between 
the  purchasing  department  and  the  supplier. 
This  is  best  tested  by  supplier  visits.  Does 
he  have  all  the  information  that  he  needs  to 
produce  a  Quality  Product?  Often  the  answer 
is  no.  Corrective  action  is  Initiated  by  the 
auditor.  It  probably  should  have  been  taken  by 
Source  Inspection  so  included  in  the  audit  is  an 
evaluation  of  file  effectiveness  of  the  source 
inspector.  A  minor  amount  of  product  re¬ 
inspection  is  performed  during  tba  audit  to 
determine  two  things  —  does  the  supplier 
conform  to  requirements  and  has  our  own 
Quality  Engineering  identilled  all  of  the  Quality 
requirenoents?  If  not,  they  are  reported  in 
the  audit  and  a  Supplier  Action  Request  (SAB) 


is  issued  to  the  purchasing  department  as 
well  as  an  Action  Request  (QA)  to  the  manager 
of  Outside  Procurement  Inspection.  These 
are  very  important  facets  to  supplier  audits. 
Does  he  have  a  Quality  system  responsive  to 
imposed  requirements,  how  does  he  control 
his  critical  processes  and  does  he  have  a 
calibration  system  which  is  effective  in  guaran¬ 
teeing  uniform  acceptability  of  the  product 
traceable  to  Ihe  National  Bureau  of  Standards 
to  name  a  few. 

Auditors  cannot  accomplish  all  of  the  desired 
results  without  a  lot  of  pre-audit  preparation. 
You  can't  send  an  auditor  to  a  supplier  to 
audit  off  the  top  of  his  head.  Accordingly,  we 
have  an  organized  Pre-Audit  Plan  which  is 
Imposed  on  the  auditor.  Such  things  as  pur¬ 
chasing  requirements,  design  data,  previous 
known  problem  areas  and  so  forth  must  be 
incorporated  in  the  Audit  Plan.  Aligned  with 
these  constraints  is  the  source  of  information. 
The  auditor  must  compile  this  information 
prior  to  scheduling  the  audit  thus  assuring 
that  he  known  what  he  is  looking  for  when  he 
gets  on  board. 

We  have  determined  the  types  of  audits  and  it 
is  to  be  anticipated  that  some  unsatisfactory 
findings  will  result.  How  are  they  fed  bade 
for  action?  We  use  a  serialized  Quality 
Assurance  Report  categorized  as  an  Action 
Request  (AR).  In  the  case  of  vendors,  it  is  a 
Supplier  Action  Request  (SAR).  It  is  directed 
to  responsible  management;  it  states  the 
problem  and  it  demands  timely  action  with  a 
space  provided  for  an  explanation  of  the  action 
taken.  These  are  logged  in  the  audit  group 
and  follow  up  is  made.  Where  action  is  not 
prompt,  a  'do  better”  slip  is  issued.  If 
action  is  still  not  forthcoming,  it  is  brought 
to  the  attention  of  successively  higher  manage¬ 
ment  for  decision.  When  the  AR  or  SAR  is 
returned  to  fiie  auditor,  the  answer  is  analyz¬ 
ed.  If  not  satisfactory,  conferences  are 
called  for  esolutlon. 

All  audit  reports  are  issued  with  a  distinctive 
cover  sheet.  As  an  attention  getter,  it  is  on 
colored  paper  so  that  it  does  not  get  mixed 


with  routine  memos  on  the  recipient's  desk. 

The  audit  group  files  a  copy  and  when  all 
actions  are  complete  and  satisfactory,  the 
auditor  applies  his  audit  stamp  with  the  word 
'Vslosed"  and  the  audit  is  filed  for  future 
reference  and  possible  review  by  the  custom¬ 
er.  To  this  end  Convair  maintains  an  open 
book  to  the  customer  and  seeks  his  review. 

To  keep  management  and  the  customer  advised, 
a  monthly  summary  is  published  of  all  activity 
conducted  during  the  month.  It  portrays  the 
number  and  types  of  audits,  the  percent  non¬ 
conformance  for  the  month,  and  the  percent 
non-conformance  year  to  date.  Also  Included 
is  a  goal  which  is  developed  yearly  based  on 
past  performance  and  by  conference  with  the 
producing  departments. 

Our  NASA,  AF,  Navy  and  Commercial  cus¬ 
tomers  have  audit  requirements  Included  in 
their  contractual  documents.  Notable  is  NASA 
document  NHB5300. 4  which  in  paragraph 
1B205  imposes  the  responsibility  upon  the 
contractor  to  conduct  internal  audits  wl,th  an 
Independent  team  of  specialists.  Customer 
administrative  documents  impose  upon  them 
the  responsibility  of  auditing  the  contractor. 
Most  customers  satisfy  this  requirement  by 
conducting  "over  the  shoulder"  audits.  This 
audit  is  conducted  in  coqjunction  with  the  con¬ 
tractors  on  going  audit  activity. 

Previously  our  customers  dispatched  a  team 
to  our  facility  several  times  a  year  and  with 
multiple  customers  this  resulted  in  consider¬ 
able  disruption  in  normal  activity  and  was  in 
effect  time  consuming  for  the  customer  and 
the  contractor.  With  the  'bver-the-shoulder" 
concept  the  efficiency  of  both  customer  and 
contractor  has  been  measurably  Improved. 

The  customer  reviews  the  schedule  of  audits 
and  decides  what  he  wishes  to  participate  in 
on  a  timely  basis.  He  has  the  advantage  of 
prepared  audit  plans,  the  results  of  prior 
audits  and  access  to  failure  history.  He 
flinctlons  as  an  actual  member  of  the  con¬ 
tractor's  audit  team  still  maintaining  an 
independent  Judicial  attttude.  His  independent 
findings  are  entered  into  the  audit  discrepancy 


report  and  action  requests  are  generate*]  in 
the  normal  fashion.  He  conducts  an  audit  de¬ 
briefing  to  discuss  his  opinion  of  how  the  audit 
was  conducted  and  to  present  any  significant 
findings  he  has  gleaned  over  and  above  those 
of  the  contractor's  audit  personnel.  The 
contractor's  audit  team  follows  up  on  cor¬ 
rective  actions  and  is  obligated  to  report  to  the 
customer.  The  customer  has  the  additional 
option  of  doing  two  things.  He  can  request 
special  audits  in  which  he  may  or  may  not 
participate,  or  he  can  bring  in  his  own  audit 
team  and  conduct  an  audit  Independent  of  the 
contractor. 

Where  is  the  pay  off?  There  are  definite 
benefits  to  over-the-shoulder  audits. 

There  are  several  significant  factors  worth 
your  attention.  The  review  of  Convair 's  pre  - 
vious  audits  and  adequacy  of  the  corrective 
action  taken;  the  free  unannounced  area  of 
participation;  the  review  of  the  contemplated 
audit  plan;  the  establishment  of  findings  with 
attendant  planned  corrective  action  and  last 
but  not  least,  the  helpful  information  impart¬ 
ed  to  Convair  management  in  the  debriefing 
which  enables  us  to  be  more  sensitive  to 
customer  desires. 

e  It  is  a  cost  effective  way  for  the  customer 
to  evaluate  the  Contractor's  Quality. 

«  It  gives  him  foe  advantage  of  utilizing 
trained  contractor  personnel-seeing  eye 
dogs  as  it  were. 

e  It  results  in  increased  sensitivity  to  cus¬ 
tomer  requirements  for  product  quality, 
reliability  and  cost  reduction. 

e  It  satisfies  the  customer's  self  imposed 
Policy  Directives  for  Contractor  audits 
in  a  most  economical  fashion. 

e  The  audits  detect  and  report  system 
deficiencies  to  management. 

e  Producers  and  Inspectors  are  mo  re 
attentive  to  product  quality. 

e  Timely  detection  of  product  deficiencies. 


•  Cost  reduction  by  elimination  of  repetitive 
rejections. 

We  have  seen  that  the  technique  of  auditing  for 
Ihe  control  of  Quality  is  not  an  entirely  new 
adventure.  It  has,  however,  come  into  pro¬ 
minence  in  recent  times  as  a  management 
technique  leading  to  a  higher  level  of  quality. 

It  is  most  effective  only  when  the  audit  re¬ 
ports  to  the  highest  level  of  management  and 
is  devoid  of  productive  pressures. 

Specific  types  of  audits  tailored  to  the  demands 
of  the  product  must  be  utilized  such  as  pro¬ 
duct  reinspection,  process  audits,  test  audits, 
calibration,  system  and  procedures  and  audits 
of  specific  problem  areas  which  have  been 
identified.  It  must  not  be  confined  to  inrhouse 
activity,  but  must  also  encompass  suppliers 
and  their  sources. 

It  is  very  important  that  audits  be  carefully 
planned  using  audit  plans  established  as  a 
result  of  research  of  design  and  system 
requirements. 

Participation  by  the  customer  in  over-the- 
shoulder  audits  should  be  encouraged  for  it  is 
cost  effective  and  adds  to  the  discipline  when 
conducted  in  an  open  and  fair  cooperative 
manner.  The  success  of  the  program  is  vest¬ 
ed  in  management  support,  comprehensive  and 
objective  audit  reporting  with  an  adequate 
system  of  corrective  action  response  and 
close  out. 

Our  customers  believe  that  we  have  one  of  the 
best  audit  programs  in  the  industry  and  are 
completely  satisfied  in  the  over-the-shoulder 
audit  concept.  The  attached  letter  from  one 
of  our  customers  is  witness  to  this  fact. 


Jain  F.  Kcwwdy  SpmCwiar 
KannadySpicaCanlar.Flondi 
32899 
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General  Drynamica  Corporation 

Comralr  Aerospace  Divlaion 

Attn:  Mr*  S*  B*  Chamberlain 

1*VP  Asaurance  Manager 

Dept  BSO-0 

P.O.  Box  1128 

San  Diego.  CA  82112 

Snhiect:  GD/C  Quality  Assurance  Audit  ol  ETH  (^rations 


M  December  1976  and  tarice  in  1977,  personnel  from  KSC  Quality 
Assurance  Survey  Branch  participated  in  GD/C  audits  as  observers 
in  Over-the-Shoulder  audit  approach.  These  thorough  and  compre¬ 
hensive  audits  and  audit  reports,  along  with  the  audit  report  review 
by  the  Chief  of  KSC  Expendable  V  ehicle  Quality  Assurance  Office. 
Mr,  G.C.  Mayer,  satisfactorily  meets  all  the  requirements  of  KSC 
yearly  survey  schedule.  With  such  a  proficient,  competent  audit, 
tt  is  possible  to  further  cost  savings.  Therefore,  personnel  from 
my  office  will  no  longer  participate  in  these  audits  or  request  GD/C 
audit  reports.  GD/C  audits  and  the  review  made  by  Mr.  Mayer, 
as  previously  stated,  meet  KSC  yearly  survey  schedule  require¬ 
ments.  B  there  are  any  quality  assurance  areas  that  require 
deUUed  scrutiny  by  GD/C  audit  team.  1  will  get  in  touch  with 
you  directly  or  thrtragh  Mr.  Mayer. 


1  commend  GD/C  Qualify  Assurance  Audit  OfBce  for  a  job  well 


.'John  R.  Atkina 
''  Acting  Chief, 

Product  Assurance  Office 
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PRIME  CONTRACTOR  FLOW  DOWN  REQUIREMENTS 


Louis  R.  Criez 
Subcontracts  Manager 
Space  Transportation  Systems 
Boeing  Aerospace  Company 


As  you  can  see  from  the  topics  of  dis¬ 
cussion  in  this  conference,  mission 
assurance  requirements  and  activities  are 
found  in  almost  every  aspect  of  our 
business.  The  pressures  of  schedules, 
budgets,  practicality,  experience, 
ability  to  adapt  to  new  ideas  and  methods 
and  more,  all  weigh  heavily  on  the  out¬ 
come  of  our  efforts  and  the  success  of 
a  particular  program.  I  would  like  to 
discuss  this  subject  with  you  as  seen 
from  the  subcontracting  function.  I  will 
review  the  prime  contract  flow  down 
requirements  to  the  subcontractor,  how 
this  is  accomplished  and  then  go  over  a 
few  of  the  major  problems  we  have 
encountered  over  the  past  years  which 
are  related  to  mission  assurance.  I 
would  hope  to  leave  you  with  a  few 
thoughts  for  further  discussion  in  this 
workshop  and  perhaps  an  idea  or  two  on 
how  things  can  be  improved. 

Where  do  the  Requirements  Come  From 

The  basic  derivation  of  requirements  has 
not  changed  much  over  the  years.  As  you 
might  expect,  the  customer's  request  for 
quotation  is  the  repository  of  his 
requirements.  The  prime  contractor 
extracts  these  requirements,  interprets 
them  through  discussions  with  the 
customer,  adds  any  company  requirements 
which  are  necessary  to  accomplish  the 
task  and  flows  the  requirement  to  the 
subcontractor  through  the  subcontract 
request  for  proposal.  ,The  requirements 
are  then  interpreted  by  the  subcontractor, 
discussed  with  the  prime  contractor  and 
used  as  the  basis  for  the  subcontract 
proposal.  I  have  selected  some  of  the 
flow  down  requirements  from  a  high 
reliability  program  for  your  information. 
The  right  hand  column  provides  a  comparison 
to  a  high  reliability  commercial  program. 
You  can  see  there  are  many  controls. 

This  discussion  will  highlight  the  parts 
materials  and  process  requirements. 

Chanaes  occur  during  prime  contract  nego¬ 
tiations  and  interpretations  are  refined  as 
work  begins  and  customer's  desires  become 
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more  definitive. 

So,  program  requirements  come  from  .  .  . 

a.  Request  for  Quotation 

b.  Prime  Contractor  Requirements 

c.  Various  Communications  with  the 
Customer 

Definitized  Prime  Contract 


d. 

e. 


Government  Interpretation  of 
Definitized  Prime  Contract 


What  are  These  Requirements 

The  contract  requirements  can  be  put  into 
three  groupings,  which  are: 

a.  Legal  -  This  group  is  composed  of 
items  such  as  Buy  American  Act, 
Rights  in  Data,  Affirmative  Action, 
Defense  Acquisition  Regulations 
and  so  on.  Boiler  plate  type 
things. 

b.  Technical  -  Consisting  of  the 
Military  Specifications,  Design 
Requirements,  Reliability,  and 
so  forth. 

c.  Administration  and  Management  - 
Which  include  items  like  Cost 
Reporting,  Work  Breakdown 
Structure,  Schedules,  and  Data 
Requirements. 

System  effectiveness  requirements  are 
included  in  the  technical  and  management 
groups. 

Now  How  are  These  Reflected  in  Subcontracts 

The  requirements  are  given  to  the  sub¬ 
contractor  in  the  Statement  of  Work,  the 
Terms  and  Conditions  and  the  Procurement 
Specification.  The  Procurement  Specifica¬ 
tion  is  called  by  many  names.  It  is  the 
document  that  details  the  design  and  test 
requirements.  Now  you  have  a  general 
picture  of  what  requirements  are,  where 
they  come  from  and  how  they  appear  in  a 
subcontract.  It  is  all  clear  and  straight¬ 
forward,  but  as  you  can  imagine,  this  is 
often  not  the  case. 

How  Does  This  All  Work 

A  prime  contract  having  significant  sub¬ 
contracts  usually  represents  a  task  which 
is  relatively  well  known  to  the  interested 
subcontractors.  Special  studies. 
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procurement  cycle  conceptual  phases, 
meetings,  etc.  precede  subcontract 
proposal  activity  and  awards.  Through 
this  type  of  activity,  a  general  under¬ 
standing  of  the  requirements  emerges  from 
which  many  assumptions  about  requirements 
can  be  made. 

I  will  point  out  a  few  of  the  major 
problems  and  then  discuss  them  in  more 
detail  later. 

The  first  significant  problem  related  to 
assumptions  is  that  customer  requirements 
are  not  available  when  needed.  The  next, 
as  I  see  things,  is  that  new  requirements 
are  seldom  definitized  and  that  Government 
interpretations  of  these  requirements  are 
not  adequately  planned  for.  The  next  is 
that  different  or  unusual  application  of 
specific  military  standards  and  specifica¬ 
tions  is  generally  misunderstood.  The 
tree  of  reference  documents  has  become 
so  complex  and  interwoven  that  time  does 
not  allow  adequate  review  and  understanding. 
Finally,  is  the  perennial  problem 
resulting  from  the  preceding  items;  the 
cost  estimates  are  reflective  of  the 
assumptions,  understandings  and  misunder¬ 
standings  which  may  or  may  not  be  res¬ 
ponsive  to  the  task  as  envisioned  by  the 
customer.  Now,  let  me  go  a  little  deeper 
into  these  problems  with  requirements  flow 
down.  The  first  problem  mentioned  was, 
customer  requirements  not  available  when 
needed.  Subcontractor  pricing  must  precede 
prime  contractor  pricing  and  the  sub¬ 
contractor  needs  no  less  time  to  prepare 
his  proposal  than  the  prime  contractor. 

To  provide  this  time,  subcontractor 
activities  begin  very  early  and  it  has 
consistently  been  my  experience  that  this 
activity  was  started  substantially  ahead 
of  receipt  of  the  customer  RFP.  The  prime 
contractor  establishes  the  flowdown  based 
on  the  best  information  he  has  .  .  . 
assumptions  are  made.  Granted,  there  is 
information  on  which  to  base  the 
assumptions,  but  the  only  acceptable  way 
of  going  is  to  have  the  customer  require¬ 
ment  in  time  to  use  them  for  quoting 
the  job. 

The  next  problem  mentioned  was  that  new 
requirements  were  seldom  understood 
(definitized)  and  that  government 
interpretation  of  these  new  requirements 
were  seldom  planned  for.  I  am  not  saying 


that  new  requirements  are  totally  mis¬ 
understood.  What  I  mean  is  that  the 
timing  from  receipt  of  a  requirement  or 
the  determination  that  there  is,  in  fact, 
a  requirement,  and  what  the  customer 
intends,  and  the  subcontractor's  proposal 
is  too  short  to  allow  adequate  discussion 
and  understanding.  In  the  proposal  rush, 
unanswered  questions  are  handled 
inadequately.  In  many  cases  the  require¬ 
ments  become  definitized  during  the 
process  of  trying  to  do  the  work.  Pricing 
and  schedule  agreements  are  based  on 
assumptions,  interpretations  and  under¬ 
standings  that  may  or  may  not  be  correct. 

Since  this  is  such  a  significant  area,  I 
will  go  into  more  detail  with  an  example. 

In  recent  experience,  we  were  required  to 
comply  with  SAMS0-STD-73-2C  "Electronic 
Parts,  Materials  and  Processes  for  Space 
and  Missile  Applications,  Standard  Control 
Program  for."  We  at  Boeing  were  familiar 
with  the  document  as  were  the  major  sub¬ 
contractors.  This  familiarity  resulted 
in  the  following  expectations; 

a.  Compliant  specification  could  be 
derived  by  modifying  existing 
specifications. 

b.  Technical  problems  could  be 
resolved  by  deviations  and  waivers 
to  73-2C  or  to  the  approved  parts 
specifications.  (Didn't  really 
believe  the  customer  would  enforce 
the  requirements  since  waivers 
and  deviations  were  readily 
granted  on  previous  programs). 

c.  73-2C  parts  would  be  available 
when  needed. 

d.  Budgets  predictions  at  the  prime 
level  and  subcontractor  level 
would  be  adequate. 

None  of  these  expectations  were  realized. 
Preparation  of  parts  specifications  and 
obtaining  customer  approval  was  a  massive 
effort  far  exceeding  budgets.  Approval 
cycles  for  parts  specifications  took  many 
months  and  some  over  a  year.  Negotiation 
of  approved  specifications  with  the  parts 
sources  took  longer  than  expected  and  lead 
times  for  73-2C  parts  did  not  support 
program  schedules.  Parts  suppliers  found 
the  73-2C  business  to  be  impacting  other 
more  profitable  business  and  schedules 


deteriorated  further.  Substitute  parts 
were  required  to  support  hardware  fab¬ 
rication  schedules.  The  customer  was 
unwilling  to  deviate  from  their 
interpretation  of  the  requirements  and 
on  matters  of  technical  interpretation, 
consistently  required  the  course  of  action 
which  had  greatest  impact  on  costs  and 
schedules.  The  basic  prime  and  sub¬ 
contractor  assumptions  missed  the  reality 
of  the  situation  and  specification 
preparation  and  approval,  parts  costs, 
parts  expediting,  substitute  parts  and 
other  activities  to  obtain  approved  parts 
to  support  the  program  devastated  budgets 
and  program  schedules.  Another  considera¬ 
tion  is  that  compliance  with  this  type 
requirement  is  generally  beyond  the 
capability  of  small /disadvantaged 
business  concerns. 

Different  or  unusual  application  of 
specific  standards  and  specifications  are 
generally  misunderstood.  During  the  rush 
of  proposal  preparation,  referenced 
compliance  documents  can  be  and  are  over¬ 
looked.  This  can  best  be  demonstrated 
by  another  example  from  recent  experience. 
MIL-P-28809  "Printed  Wiring  Assemblies" 
is  a  specification  called  out  by  MIL- 
STD-454  which  is  in  turned  called  out 
by  SAMS0-STD-73-2C.  Since  the  sub¬ 
contracted  items  were  to  be  qualified  to 
MIL-1540,  qualifying  the  printed  wiring 
assemblies  to  MIL-P-28809  was  assumed 
not  to  be  required,  when  in  fact,  it  was, 
and  there  was  no  money  budgeted  for  the 
task. 

From  this  discussion  thus  far,  some  of 
the  difficulties  in  preparing  a 
responsive  proposal  should  be  apparent. 

The  problems  discussed  relate  particul¬ 
arly  to  high  reliability  programs  and 
the  impact  is  most  clearly  seen  in  the 
final  problem  mentioned.  That  is,  the 
cost  estimates  and  schedule  conmitments 
refl ect  understandings/mi sunders tandi ngs . 
Simply  stated,  we  have  found  that  efforts 
to  keep  costs  down  on  high  reliability 
programs  enhance  the  probability  of 
inadequate  cost  and  schedule  estimates. 

No  one  wants  to  pay  the  price.  There  is 
no  practical  relationship  between  the 
costs  of  a  very  high  reliability  program 
and  the  usual  mil -spec  type  program  and 
yet  the  comparisons  are  made  with  result¬ 
ant  inadequate  budgets. 


I  have  exposed  you  to  a  few  high  points 
of  this  coirplex  business  from  the  view¬ 
point  of  subcontract  management  at  the 
prime  contractor  level.  Suggestions  to 
minimize  the  impact  of  system  effective¬ 
ness  requirements  are: 

1.  Standardize  requirements;  i.e., 
fix  mil -specs  rather  than  invent 
new  ones. 

2.  Identify  the  requirements  early 
and  clearly. 

3.  Conduct  a  bidders  briefing  with 
the  customer  and  all  prime  and 
subcontract  bidders  present  to 
outline  and  explain  any  new  and 
unusual  requirements.  Convince 
the  bidders  that  this  is  really 
what  is  required  and  then  be 
willing  to  pay  the  price. 

4.  Place  a  customer  73-2C  adminis¬ 
trator  on  site  in  the  prime 
contractor's  plant  to  handle 
parts,  materials  and  processes 
problems  in  real  time. 

This  is  a  brief  look  at  the  requirement 
flow  down  situation.  Now  let's  look  at 
the  recipient  of  these  requirements,  the 
subcontractor,  and  his  problems  in 
producing  hardware  while  meeting  the 
requirements. 


MISSION  /ISSURAIICE  CONFEHEUCE 
SUBCONTRACT  MANAGEMENT 


REQUIREMENTS  FLOW  DOWN 
DERIVATION; 

REQUEST  FOR  QUOTATION 
PRIIIE  CONTRACTOR  REQUIREMENTS 
COMMUNICATIONS  WITH  CUSTOMER 
DEFINITIZED  PRIME  CONTRACT 

CUSTOMER  INTERPRETATION  OF  DEFINITIZED  PRIME  CONTRACT 


fllSSlON  ASSURANCE  COflFERENCE 
SUBCONTRACT  flANAGEMENT 

REQUIREMENTS  FLOW  DOWN 
GROUPINGS: 

.  LEGAL 
.  TECHNICAL 

.  ADMINISTRATION  &  MANAGEMENT 


niSSIOH  ASSURANCE  COflFEREflCE 
SUDCONTRACT  flANAGEHENT 

REQUIREMENTS  FLOW  DOWN  PROBLEMS; 

CUSTOMER  REOUIREriENTS  HOT  AVAILABLE  WHEN  NEEDED 

NEW  REQUIREMENTS  SELDOM  DEFINITIZED 

GOVERNMENT  INTERPRETATIONS  NOT  ADEQUATELY  PLANNED  FOR 

DIFFERENT  OR  UNUSUAL  APPLICATION  OF  SPECIFIC  MILITARY 
STANDARDS  t  SPECIFICATIONS  MISUNDERSTOOD 

COST  ESTIMATES  S  SCHEDULE  COMMITMENTS  REFLECT 
ASSUMPTIONS  AND  MISUNDERSTANDINGS 
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niSSIOH  ASSURANCE  CONFERENCE 
SUBCONTRACT  MANAGEMENT 

REQUIREMENTS  FLOW  DOWN 
SUBCONTRACT: 

.  TERMS  t  CONDITIONS 
.  PROCUREMENT  SPECIFICATION 


•issio':  Ass"R/‘srr  coNrcurcr 

SUBCONTRACT  MAVAGCMENT 


STATEMENT  OF  WORK 


tllSSIOn  ASSURANCE  COtlFERENCE 
SUBCONTRACT  HANAGEHENT 


73-2C  PERSPECTIVE 

STRAIGHT  MIL  SPEC  TYPE  lUS  73-2C 


PROGRAM  ELEMENT 

PROGRAM 

PROGRAM 

PRXRAfl 

PARTS  COST 

(OVERALL) 

X 

SA 

8X 

(IC'S) 

X 

GX 

16X 

PMP  ENGINEERING 

X 

2.2X 

4. NX 

PMP  PROCUREMENT  SCHEDULING 

X 

2.5X 

3X 

MANUFACTURING  EFFORT 

X 

1.05/ 

I.IX 

0.  A.  EFFORT 

X 

1.5X 

3X 

MISSION  ASSURANCE  COflFEREHCE 
SUBCONTRACT  MANAGEMENT 


SUGGESTIONS  FOP.  IMPROVEMENT 

.  STANDARDIZE  REBUIREMENTS,  I.E.  FIX  MIL-SPECS 
DON'T  INVENT  REFIACEMENTS 

EARLY  AND  CLEAR  IDENTIFICATION  OF  REQUIREMENTS 

.  CONDUCT  BIDDERS  BRIEFING  TO  IDENTIFY  AND  EXPUIfl 
NEW  AND  UNUSUAL  REQUIREMENTS 

.  FOR  REQUIREMENTS  SUCH  AS  73-2C  PUT  CUSTOMER  REPRESENTATIVE 
ON  SITE  IN  PRIME  CONTRACTOR'S  PUNT  TO  HANDLE  PMAP 
PROBLEMS  IN  REAL  TIME 


SUBCONTRACTOR  PROBLEMS  WITH  PRIMES 
AND  COMPONENT  SUPPLIERS  UNDER 
SAMSO  73-2C 

Sam  Panel! a 
Ed  Thibodeau 
Hamilton  Standard 


What  Is  Good  About  73-2C 


•  ESTABUSHES  THE  MINIMUM  REQUIREMENTS  FOR  A  PMP  CONTROL 
AND  STANDARDIZATION  PROGRAM  FOR  SPACEBORNE  EQUIPMENT. 


•  RESTRICTS  THE  USE  OF  PMP  WHICH  HAVE  BEEN  DEFINED  AS 
REUABIUTY  RISKS 


•  ESTABLISHES  UNIFORM  SCREENING  REQUIREMENTS 


#  ESTABLISHES  A  UNIFORM  DERATING  POUCY 


Lot  Joopardy  Scroetis 

•  SCANNING  ELECTRON  MICROSCOPY  (SEM) 

•  PARTICLE  IMPACT  NOISE  DETECTION  (PINO) 

•  PERCENT  DEFECTIVE  ALLOWABLE  (PDA) 

•  DESTRUCTIVE  PHYSICAL  ANALYSIS  (DPA) 

•  LOT  QUALITY  TESTS  (QUAUFICATION  OR  QUALITY  CONFORMANCE) 


Intorfaco  With  Primo  Contractor 


Working  With  73-2C 


•  DOCUMENT  WAS  NOT  MATURE 

•  DOCUMEKTATtON  REQUIRED  TO  GET  PMP  APPROVAL  WAS  EXCESSIVE 

•  APPROVAL  CYCLE  TIME  FOR  NSPAR's  AND  SPECIFICATIONS  WAS 

Excessivz 

•  SUBCONTRACTORS  WERE  NOT  ALLOWED  TO  USE  MATERIALS  AND  PROCESSES 
WITH  WHICH  THEY  ALREADY  HAVE  EXPERIENCE  WITHOUT  DETAILED  JUSTIF* 
ICATXM 

•  KTH  OF  EXISTING  DRAWINGS  FOR  STANDARD  MATERIALS  WAS  NOT 
A  JTHORi  2ED 

•  APPROVED  VENDOR  t  tSTS  AND  COORDINATED  PROCUREMENT  MAY  BE  A 
If^L  CANTI  i'R'J^T)  PROBLEM 

•  SOME  REQU'.REMEKfS  ARE  FOLVO  IN  FOURTH  TIER  DOCUMENTS 


•  3AMSO-~STD  73-4  NOT  INCORPORATED  INTO  THE  PPSL 

•  USE  OF  THE  SPECIFICATION  APPROVAL  CYCLE  FOR  OTHER  THAN 
WHAT  rr  WAS  INTENDED 

•  INFLEXIBILITY  IN  ADMINISTERING  THE  PROGRAM  REQUIREMENTS 

•  ENFORCEMENT  OF  INEFFECTIVE  PART  SCREENS 


ineffective  Part  Screens 

•  X-«Ay  TANTALUM-TANTALUM  WET  SLUG  CAPACITORS 


Evolution  Of  Screening  Requirements 


•  N-RAY  TEMPERATURE  COMPENSATED  CERAMIC  CAPACITORS 


•  SAMSO-STD  73-2C 

•  SAMSO-LVGS  77-005 

•  CHANGES  TO  SAMSO-LVGS  77-005 

•  PROPOSED  CHANGES  TO  SAMSO-LVGS 
77-005  FOR  THE  PRODUCTION  PHASE 
OF  THE  PROGRAM 

•  OPTION  TO  USE  ERM1L  CAPACITORS  ANO 
RESISTOR.  TCP-0t21 


2  SEPTEMBER  1075 
1  JULY  1977 
28  FEBRUARY  1979 
28  MARCH  1980 

9  APRIL  1980 


•  PERFORM  FIST  ON  TEMPERATURE  COMPENSATED  REFERENCE 
DIODES 


Recommendations 


•  A  SYSTEM  FOR  FASTER  TURN  AROUND  ON  NSPAR  ANO  SPECIFICATIONS 
APPROVAL  SHOULD  BE  IMPLEMENTED 


Buying  Parts  Tu  73-2C 

•  SOME  VENDORS  REFUSE  TO  SUPPLY  PARTS 

•  NO  VENDOR  WILL  ACCEPT  IT  FOR  SMALL  ORDERS 

•  LOT  REJECTION  REQUIREMENTS  INCREASE  THE  COST  SIGNIFICANTLY 


•  FLEXIBILITY  IN  ALLOWING  SUBCONTRACTORS  TO  USE  MATERIALS  ANO 
PROCESSES  WITH  WHICH  THEY  ALREADY  HAVE  EXPERIENCE 

•  USE  OF  EXISTING  DRAWINGS  FOR  STANDARD  MATERIALS  SHOULD  BE 
AUTHORIZED 

•  ALL  IMPOSED  SPECIFICATIONS  SHOULD  BE  DEFINED  IN  7J-2C 

•  SUBCONTRACTORS  SHOULD  BE  FREE  TO  USE  MORE  OF  THE  PARTS 
FROM  THE  P-99  MONITORED  UNE 

•  SUBCONTRACTORS  SHOULD  BE  REQUIRED  TO  BUY  EACH  PART  TYPE  FROM 
TWO  SOURCES 

•  9EM  SHOULD  BE  DONE  BY  THE  CUSTOMER.  NOT  BY  THE  VENDOR 


•  THE  NUMBER  OF  VENDORS  FROM  WHICH  COMPONENTS  CAN  K 
BOUGHT  IS  restricted 


•  DPA  SHOULD  BE  PERFORMED  AS  EARLY  IN  THE  PROGRAM  AS  POSSIBLE 


Major  Material  Problems 

I.  COST 

•  RISING  COST  DUE  TO  73-2C  SCREENS 

•  SMALL  PRODUCTION  QUANTITY 

II.  AVAILABILITY 

•  VENDOR  LEAD  TIME 

•  SUCCESS  IN  MEETING  SPEC  REQUIREMENTS 


Material  Cost  FSD  versus  Production 
(Excluding  Lot  Charges) 

COST  $ 


Material  Cost  FSD  versus  Production 
(Including  Lot  Charges) 


Purchase  Larger  Buy 


ADVANTAGES 

A.  UNIT  COST  REDUCED 

B.  PARTS  AVAILABILITY 

C.  SUPPORT  COST  REDUCED 

D.  NEGOTIATION  POSITION 
DISADVANTAGES 

A.  LARGER  CAPITAL  TIED  UP 

B.  INVENTORY  VERSUS  INVESTMENT 


73-2C  Availability 


Suggestions 


HiSTOWY 

VENDOR  PROMISES  ON  ACTIVE  OEVICES  PRIOR  TO  GROUP  B 
AVERAGED  30  WEEKS 

•  ONLY  l5%OF  THE  PARTS  CAME  IN  ON  TIME 

•  average  was  so  weeks 

•  SOME  PARTS  ALMOST  TWO  YEARS  LATER  HAVE  NOT 
RECEIVED  COMPLETE  ORDER 

PRODUCTION 

•  VENDOR  QUOTING  LONGER  LEAD  TIMES  INITIALLY 


•  OFFER  VENDORS  INCENTIVES 

•  OFFER  VENDORS  PREMIUM 

•  PROVIDE  EDUCATIONAL  AWARENESS  PROGRAM 

•  NEGOTIATE  OX  PRIORITY  RATING 
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•  OUAUeOUnCINOOMXM 

•  STAirr  MULTiPLC  U0T8  (oimncKr  mctauzation) 
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(a)  BMNmCB  WILL  NO  LONKB  AOMB  TO  KM  mEOUCBTB  ON  FNOMS 


SUMMARY  OF  ISSUES 
AND  RECOMMENDATIONS 


SUBCONTRACT  MANAGEMENT 


SUMMARY  OF 

ISSUES  AND  RECOMMENDATIONS 


1.  Subcontract  Management 
Communications 

Issue 

Improve  communication  between 
the  customer  and  subcontractor. 

Recommendations 

oPre  and  post  award  seminars 
(customer,  contractor,  geo¬ 
graphical)  awareness  of  program 
objectives.  Awareness  of  con¬ 
tract  content  and  meaning.  Subs 
attend  with  prime. 

o  Prime  contractural  direction 
that  these  meetings  be  held. 

oSub  contractors  attend  special¬ 
ized  meetings  and  review, 

(i.  e.  ,  PMPCB)  both  at  the  con¬ 
tractor  and  at  rotated  locations. 

o  Clear  contractual  definition  of 
customer  entry  to  subs.  SPO 
must  notify  prime  of  visits  to 
subs. 

D  Continue /expand  mission  assur¬ 
ance  conferences  to  encourage 
subcontractor  participation. 

o  Invite  all  the  subs  to  visit  the 
prime  to  check  commonality,  co¬ 
ordinated  purchases,  a  team 
effort  for  the  entire  procurement 
process,  getting  the  subs  talking 
to  each  other,  to  understand 
lessons  learned,  to  explain  flow- 
downs. 

2.  Subcontract  Requirements 
Issue 

Customer  requirements  are  pro¬ 
liferating  while  industry  is  striv¬ 
ing  to  be  more  autonomous  and 


less  controlled  (monitored'.  Need 
understanding  and  generally  same 
approach  to  implementing  same 
requirements. 

Recommendations 

b  Conduct  post  award  reviews  with 
subs  for  in  depth  review  of  appli¬ 
cation  of  requirement  to  subs. 

o  Highlight  reason  for  origination 
of  requirement  in  pre-award  con¬ 
ferences. 

o  Conduct  periodic  review  of  spec¬ 
ifications  to  modify ^cahcel  to  re¬ 
flect  current  needs  and  specific¬ 
ally  review  against  original  rea¬ 
son  for  specification. 

o  Provide  appendix  to  spec's  that 
allows  for  specific  paragraph 
by  paragraph  applicability  that 
ties  to  program  phasing. 

o  Include  customer,  contractor, 
sub-contractor,  AFPRO,  DCAS 
in  a  post  award  understanding  of 
of  requirements  and  document  re¬ 
sults  as  memorandum  of  agree¬ 
ment  as  a  part  of  contract. 

o  Re-examine  need  for  require¬ 
ments  in  specs.  Requirements 
are  usually  input  to  solve  a  pro¬ 
blem,  but  the  problem  may  be 
non-existent  now  or  greatly  mod¬ 
ified  by  current  technology. 

3.  Subcontract  Program  Management 
Issue 

Program  Management  (both  in¬ 
dustry  and  customer'  not  involved 
early  in  specifications.  Is  the 
boilerplate  is  what  they  want  with¬ 
out  seeing  future  costs  ? 

R  e  c  ommenda  ti  on  s 

o  Allow  more  time  for  draft  RFP's 
to  understand  "ility"  require¬ 
ments,  especially  those  large 
potential  impacts  on  cost  or 
schedule. 


o  Contractors  must  identify  (with¬ 
out  government  penalty)  those 
long  lead  items  which  preclude 
contractors  from  meeting  (plan¬ 
ned  or  present)  long  lead 
schedules  -  identify  schedule  con¬ 
straints. 

o Industry  should  consider  com¬ 
petition  secondary  in  the  single 
area  of  consolidating  buys  from 
small  buys  of  long- lead  items. 

Use  AIA  or  NSIA.  Or  use 
JAMAL  (Joint  Aeronautical 
Material) . 

10.  Surveys  and  Audits 
Issue 

SPO's  do  not  realize  that  many 
Contract  Administration  Services 
(especially  APPRO' s)  already  have 
answers  to  surveys  such  as  the 
Manufacturing  Management  Pro¬ 
duction  Capability  Reviews.  But 
this  is  not  mentioned  in  AFSCR 
84-2. 

Recommendations 

oAFSCRs  be  revised  to  include 
statements  in  each  reg  that 
AFPROs  are  involved/not  in¬ 
volved  in  certain  areas.  SPOs 
do  not  normally  understand 
AFPRO  functions  and  capabilities 
this  causes  duplicate  effort  and 
wasted  time  and  money. 

o Accept  the  centralized  data  bank 
of  surveys  (CASE),  and  establish 
Quarterly  report  on  each  sub¬ 
contractor. 

oSPOs  should  be  made  aware  of 
how  many  audits  were  done  in  a 
particular  plant  over  a  period  of 
time.  Put  this  info  into  GIDEP. 

o  Contractor  self-audit  function 
should  be  the  only  contractor 
group  SPOs  audit.  Then,  if  the 
contractor  audit  group  is  rated 
excellent,  the  government  team 
won't  have  the  need  to  come  in. 


Incentivize  the  contractor  to  do 
this. 

o  When  SPOs  go  in  for  an  audit, 
they  should  spend  minimum  time 
(depend  on  the  cognizant  CAS 
organization)  emphasizing  only 
their  program-unique  aspects. 

o  Get  the  cognizant  CAS  to  certify 
t^at  areas  of  the  SPO  audit  check 
list  are  OK  -  thereby  allowing  th« 
SPO  to  maximize  his  time  in 
plant. 

oSPOs  should  get  one  person  per 
program  to  go  in-plant  -  or  a 
multi-agency  review  (like  once 
per  year'. 

11.  Subcontract  Flowdown 
Issue 

Industry  and  government  are  both 
providing  too  much  boilerplate  to 
subs  to  be  read  and  understood. 

Recommendations 

o Prime  scrub  boilerplate  before 
flowdown  to  subs. 

o  Industry  (AIA,  NSIA)  with  SBA 
conduct  study  on  most  meaningful 
flow  down  requirements.  Study 
for  all  subcontract  tiers. 

□  Industry  (AIA  &  NSIA'  should 
suggest  increasing  flexibility 
to  DAR  requirements  to  the  DAR 
council. 

12.  Subcontract  Management  -  Public 
Law  95-507 

Issues 

How  can  government  tailor  the 
application  of  public  law  95-507 
to  the  requirements  of  the  individ¬ 
ual  contracts  and  still  meet  the 
statutory  objectives  ? 


Recommendations 

oAfter  point  by  point  review  for 
areas  requiring  modifications 
have  primes/subs  and  customer 
discussions  result  in  a  letter  of 
understanding  on  specification 
implementation. 

o  PMP  boards  capa'bility  to  inter¬ 
pret  (authorityl  specs  -  publish 
output  to  program  office  for 
approval  -  shorten  the  cycle  for 
approval.  Distribut  e  agreement 
to  affected  parties,  including 
AFPRO/DCAS. 

Subcontract  Management  - 

Tailoring 

Issue 

Tailoring  fear  -  lose  to  competitor 

or  offend  the  "author''  of  the  re¬ 
quirement. 

Recommendations 

o  Provide  positive  motivation  to 
tailoring  in  RFP,  i.  e.  ,  proposal 
will  get  additional  points  for  each 
tailored  approach. 

o  Tailoring  needs  to  reflect  cost 
savings  and  the  assurance  of  no/ 
little  loss  of  performance  -  must 
sell  approach. 

o  Continue  tailoring  throughout 
draft  and  just  prior  to  best  and 
final  -  provide  as  alternate 
approaches. 

o  Customer  "ilities"  need  to  expect 
and  treat  suggested  changes  as 
good  to  both  customer  and  con¬ 
tractor. 

o  Make  tailoring  a  positive  factor 
during  RFP  phase  -  dollars, 
source  selection  criteria,  crea¬ 
tivity,  tailoring  (you  must  sell 
your  tailoring  -  why  it  impacts 
cost  and  not  reliability'. 


8.  Subcontract  Management  -  Raw 
Mate  rials 

Issues 

Inferior  raw  materials  being  re¬ 
ceived  having  poor  traceability 
(COC's',  raw  material  specifica¬ 
tions  need  to  be  updated  to  today's 
needs. 

Recommendations 

o  Raw  material  industry  must 
accept  responsibility  for  pro¬ 
viding  products  that  meet  require 
ments  and  its  associated  certifi¬ 
cations. 

o  Sample  and  destructive  test 
incoming  raw  materials. 

o  Verify  adequacy  of  Certificates 
of  Conformance  (  COC'  espec¬ 
ially  on  flight  critical  compon¬ 
ents.  COC  must  specify  pre¬ 
cisely  what  requirements  were 
met. 

o  Indust  ry  assoc  iations  establish 
review  of  material  specifications 
to  determine  adequacy  for  space. 

oindustry  associations  initiate 
communications  with  raw 
material  industries. 

9.  Subcontract  Schedules 
Issues 

Unrealistic  schedules  for  parts 
(customer  level  vs  component 
industry' . 

Recommendations 

o  Become  more  knowledgeable  of 
lead  time  changes  based  on  needs 
and  experience. 

o  Pool  industry  and  government 
lead  time  cha  rts. 
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o  Put  inducements  into  RFP  to 
encourage  contractor  high¬ 
lighting  key  cost/schedule  impact 
with  no  compromise  to  perfor¬ 
mance. 

o  Look  hard  at  standard  "ility” 
boilerplate  being  imposed  as  re¬ 
lated  to  hardware  application. 

o  Program  managers  should  be  in 
detail  loop  review  of  require¬ 
ments  being  imposed  -  especially 
being  aware  of  contractor  position. 

oNeed  to  get  to  the  project  officers 
to  listen  to  prime  on  cost  impacts 
-  not  just  have  "filtered”  thru 
their  people. 

Subcontract  Management  Risk 

Issue 

Subtier  suppliers  do  not  want  to 

meet  Aerospace  requirements 

and  do  not  accept  contracts. 

Recommendations 

oEase  the  suppliers'  lack  of  re¬ 
sources  by  primes  taking  respon¬ 
sibility  for  some  of  tasks,  i.  e.  , 
part  procurement,  qual  testing, 
etc. 

o  Dispel  fears  of  awesome  specifi¬ 
cation  requirements  by  aware¬ 
ness  meetings  that  address 
specification  applicability. 

o  Provide  incentive  compensation  to 
encourage  suppliers  to  accept 
high  risk  projects. 

o  Provide  front  end  money  to  en¬ 
courage  specific  industry  as  a 
high  risk  technology  or  cover 
long  lead  items/materials. 

o  Cost  consideration  may  not  be  the 
best  criteria  for  proposal  judge¬ 
ment.  Make  this  the  last  element 
after  technical  input. 


5.  Subcontract  Specifications 
Issues 

New  specs  (73-2C.  MIL'S- 1 546 /71 
have  tremendous  cost  impact 
that  generally  are  unknown  at  time 
of  RFP. 

Recommendations 

o Seminar  to  explain  just  what  the 
new  specs  want,  the  costs  en¬ 
tailed,  the  knowledge  of  program 
managers  enhanced  from  the 
user  point  of  view. 

o  Mutual  interpretation  between 
customer  and  industry  to  under¬ 
stand  the  new  specs  and  the 
impact  thereof  to  eliminate  fears. 

o  Define  the  title  and  the  scope  of 
each  spec  up  front  so  that  people 
can  understand  its  application 
better. 

o  Develop  a  new  spec  with  a  mat¬ 
rix  to  highlight  changes  in  the 
spec  on  what  applies  to  each 
contract. 

o  Requirement  to  collect  costs  for 
parts  requirements.  Require¬ 
ment  for  primes  and  subs  to 
estimate  the  cost  of  the  parts 
over  a  period  of  time  per  number 
needed.  Advise  program  mana¬ 
gers  of  cost  impact. 

o  Have  conferences  on  education 
exercises  for  government  and 
industry  appropriate  for  new 
special  requirements. 

6.  Subcontract  Specification 
Inte  rpretation 

Issue 

Specs  such  as  (73-2C'  have  well 
meaning  birthrites,  but  have  be¬ 
come  inflexible  to  allow  for  moH- 
fied  implementation. 


Recommendations 


o  Conduct  a  tutorial  program  for 
government  and  industry  to  ex¬ 
plore  requirements  of  95-507  and 
what  the  contractor  is  obligated 
to  do  and  exactly  when. 

oSeek  a  contract  from  the  govern¬ 
ment  to  contact  and  educate  on 
many  of  the  small  businesses  or 
possible  in  a  specified  time  and 
make  them  aware  of  aerospace 
requirements. 

o  Could  contractors  be  judged  on  a 
yearly  basis  instead  of  a  contract 
by  contract  basis  ?  Industry 
association  inputs  to  SBA  to  pro¬ 
mote  change  in  statute  95-507. 
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STATUS  OF  MIL-STD-1540  (SYNOPSIS) 

L.L.  Jepsen 

Staff  Test  and  Evaluation  Engineer 
Space  Division  (USAF) 

MIL-STD-1540A,  Test  requirements  for 
space  vehicles,  in  use  since  1974  is  in 
the  process  of  being  updated  and  revised. 

A  draft  revisions  has  ()een  circulated  on  a 
somewhat  limited  basis  for  comment.  A  lot 
of  comments  and  suggested  changes  have 
been  received  from  industry  and  Air  Force 
offices.  Areas  and  specific  topics  that 
will  most  likely  be  changed  are  presented. 
Additional  review  and  comment  opportun¬ 
ities  will  occur  before  publication,  now 
forecast  for  early  1981. 


W1L-ST11-15W  OWIfiES 
ovtim.L 

0  SHUnif  ERA  TEST  CONSIDERATIONS 

0  SEPARATE  NANAGENENT  SYSTENS/CONTROLS  FROM  TECHNICAL  REOUIRENENTS 
0  SOFTWARE  AS  .ART  OF  THE  SYSTEM 
0  DEFINITIONS  •  CLARIFY  AND  CLEAN  UP 
0  GET  RID  OF  SUBTIER  DOCUMENTS  -  GOAL 


HlL-STD-1540  CHANG^ 

TECHNICAL  AREAS 

0  DEVELOPMENT  TESTS  -  BUT  KQI  DETAILED 
0  SUBASSEMBLY  (CARD/BOARD)  -  GET  IN  OR  QUI 
0  6  B  VS  3  B  (SOME  MORE) 

0  REQUIRED  VS  OPTIONAL  DESIGNATIONS 
0  TEST  CONDITION  TOLERANCES 
0  PARTICLE  SCREENING  TESTS  (BOX  LEVEL) 

0  THERMAL  DWELL  VS  STABILIZATION 
0  THERMAL  CYCLE  -  RATE  OF  TEMPERATURE  CHANGE 
0  RADIATION/HARDENING  TESTS  -  MAYBE 


MIL-STD-IMO  CHANGES 
TECHNICAL  AREAS 

0  THERMAL  VACUUM  -  THERMAL  CYCLING  -  THERMAL  BALANCE  TESTING 
0  RETEST  REOUIREMENTS/GUIDELINES 
0  PROTOFLIGHT/BUY-ONE-FLY-ONE  TEST  PROGRAMS 
0  SEQUENCE  OF  ENVIRONMENTS 
0  REDUNDANCY  TESTING/BURN-IN 
0  BURN-IN  DURATIONS 
0  ACCEPTANCE  TEST  OF  OUAL  ARTICLES 


0  PRELAUNCH  VALIDATION  -  MAJOR  REWRITE 


THERMAL  CYCLING  SYSTEM  TEST  (SYNOPSIS) 
R.B.  Laube 

Ground  Systems  and  Test  Department 
The  Aerospace  Corporation 

This  briefing  focuses  on  the  question 
"Can  system  level  test  effectiveness  be 
inproved  by  adding  a  system  thermal 
cycling  test  ?"  Existing  data  is  somewhat 
limited  but  indicates  that  system  level 
thermal  cycling  is  an  effective  test. 
Additional  data  and  analysis  is  needed  to 
assess  tradeoffs,  if  any,  with  thermal 
vacuum  tests  and  examine  duration,  rate  of 
temperature  change,  dwell  times  (soak), 
and  other  parameters  for  the  "best"  test 
approach.  Results  of  this  analysis  should 
be  reflected  in  changes  to  MIL-STD-1540. 


SYSTEM  UVg  THERMAL  CYUiWC  TEST 


TEST  PARAMETERS 

TEMP  RANGE 
M  CYCLES  TOTAL 
OPERATING  ANOINONITOREO 
7  SOAK  CYCLES 
0  ^m\H  RATE  OF  CHANGE 
4?  DAY  TEST  DURATION 


MISSION  ASSURANCE  CONFERENCE 
TEST  EFFECTIVENESS  WORKSHOP 
THERMAL  CYCLING  SYSTEM  TEST 

R.B.  LAUBE 
APRIL  1980 


STIMULATE  PANEL  DISCUSSION 

THERMAL  CYCLING  SYSTEM  TEST  PARAMETERS 
ROLE  OF  THERMAL  CYCLING  SYSTEM  TEST 


environmental  TEST  PROGRAM  HISTORY 
15  SPACECRAFT) 


M|L-STI>>1MQA  COMPONENT  THERMAL  CYCLING 
)SY*F  TEMP  RANGE 
I  CYCLES  total 
OPERATING  AND  MONITORED 
7  HOUR  SOAK 

S  ^FIMIN  RATE  OF  CHANGE 
7  DAY  TEST  DURATION 


MIL-STCK)S4QA  SYSTEM  THEfMAL  VACUUM 

TEMP  RANGE  BETWEEN  ORBITAL  EXTREMES 
4  CYCUS 

OPERATtNG  AND  MONITORED 
I  HOUR  SOAK 

ORBITAL  RATE  OF  CHANGE 
ABOUT  I)  DAYS  MINIMUM 


COMPONENT  TEST 


SYSTEM  TEST  , ORBIT  , 


THERMAL  CYCLING  TEST  RATIONALE 


SIGNIFICANT  NUMBER  OF  FAILURES  ATTRIBUTED  TO  TEMPERATURE  STRESSES 
DURING  COMPONENT  AND  SYSTEM  TEST 


SYSTEM  THERMAL  CYCLING  TEST  CAN  APPLY  TEMPERATURE  STRESSES  FASTER. 
MORE  UNIFORMLY.  AND  MORE  ECONOMICALLY  AT  AMBIENT  PRESSURE 


SYSTEM  THERMAL  CYCLING  ISSUES 


SUMMARY  OF  INDUSTRY  SURVEY  DATA 

TYPE  AND 
SIZE  OF 

COMPANY  DATA  SAMPLE 


270  RADAR  AUG. 
21  TRANSPONO 


TEMPERATURE  RANGE 

ALL  COMPONENTS  REACH  ACCEPTANCE  TEMPERATURE  EXTREA4ES 

HOW  MANY  CYCLES  IS  ENOUGH 
FAILURE  FREE  CYCUS 

SOAK  CYCLES 

ARE  THERE  FAILURES  MANIFEST  ONLY  AT  TEMPERATURE  EXTREMES 

TEMPERATURE  RATE  OF  CHANGE 

IS  A  RAPID  CHANGE  RATE  MORE  EFFECTIVE 

IS  A  RAPID  CHANCE  RATE  FEASIBLE  -  WORTH  THE  COST 

MOISTURE  PROBLEMS 

HOW  SHOULD  SYSTEM  THERMAL  CYCLING  TEST  BE  SPECIFIED  IN 
Mll-STO-IMOB 
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ROLE  OF  THtRMAl  CYCIING  SYSTEM  TEST 


REPLACEMENT  OF  THERMAL  VACUUM  TEST  NOT  FEASIBLE 

VERIFICATION  OF  THERMAL  CONTROL  SYSTEMS.  CORONA. 
MULTIPACTION.  OIEUCTRIC  CHARACTERISTICS  AND  OUTGASSING 


CAN  MIL-STD-1S40  THERMAL  VACUUM  TEST  BE  TRUNCATED  WITH  A 
THERMAL  CYCLING  TEST 

RUN  ALL  THERMAL  VACUUM  TESTS  WITHIN  THE  TIME  FOR  TWO 
THERMAL  BALANCE  TESTS 


COSTS  OF  THERMAL  VACUUM  VS  THERMAL  CYCLING 
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CURRENT  BOEING  EXPERIENCE  WITH 
MIL-STD-1540A  ON  COMPONENT  TESTS  (SYNOPSIS) 


Mr,  Herbert  McDaniel 
Boeing  Aerospace  Co. 

Payload  System  Assurance  Manager  -  RSLP 

Boeing  Is  In  the  process  of  developing  and 
testing  an  upper  stage  space  vehicle. 

Over  250  component  level  tests  have  been 
accomplished  In  accordance  with  MIL-STD- 
1540A.  This  represents  about  30  percent 
of  the  planned  component  qualification 
program.  Only  25  failures  have  occurred 
to  date  with  4  of  these  attributed  to  test 
procedural  errors.  The  findings  based  on 
this  testing  bring  up  the  following  issues 

1)  Sequence  Changes 

Thermal  cycling  tests  were  very 
effective  in  finding  failures  on  one 
type  of  component.  Should  thermal 
cycling  run  before  the  other  test 
environments? 

2)  Burn-in  Time 

All  failures  found  occurred  during 
the  first  few  thermal  cycles.  Should 
the  300  hours  of  burn-in  time  be 
reduced? 

3)  Thermal  Vacuum  Acceptance 

No  failures  occurred  in  thermal  vacuum 
when  thermal  cycling  was  performed 
first.  Should  thermal  vacuum 
acceptance  testing  be  made  optional 
on  low  voltage  components? 

4)  Tolerances 

Waivers  were  approved  because  the  ±  3 
db  tolerance  on  pyroshock  could  not 
be  held.  Should  MIL-STD-1540  be 
revised  to  widen  the  tolerance  to  one 
the  test  community  can  make? 

5)  Pyro  Screening 

Pyrotechnic  shock  followed  by  low 
level  vibration  bursts  was  not  effec¬ 
tive  in  finding  failures.  Should 
this  test  be  added  to  MIL-STD-1540 
as  proposed? 


ji  . 
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TEST  EFFECTIVENESS  SESSION: 

ROLE  OF  AUTOMATED  TESTING  AS  REGARDS 
MISSION  ASSURANCE 


G.T.  Yamada 

Lockheed  Missiles  &  Space  Co., Inc. 
Space  Systems  Division 


Abstract 


Current  Automated  Test  Systems  (ATS)  have 
provided  effective  testing  under  cost  and 
schedule  restrictions.  It  has  effectively 
provided  expanded  test  capabilities  in 
response  to  three  primary  requirements: 

(1)  increased  Unit  Under  Test  (UUT) 
complexity;  (2)  increased  safety  require¬ 
ments  for  UUT,  test  system,  and  personnel; 
and  (3)  kept  pace  with  technological 
advances.  It  is  also  known  that  ATS  have 
suffered  many  developmental  problems  (cost 
and  schedule  impacts)  and  operational 
problems  (performance  and  reliability). 

Most  general  scenarios  of  the  future 
indicate  quantum  jumps  in  space  mission 
requirements,  use  of  advanced  microtechno¬ 
logy  and  material  technology  products 
(VLSI,  VHSIC,  composites),  and  safety 
requirements.  The  resultant  increased  need 
for  higher  performance  in  ATS  magnifies 
increasing  ATS  problems  in  development  cost 
and  schedule  and  the  availability  of 
qualified  ATS  personnel.  Productivity 
problems  in  test  operations  will  be 
increased  if  these  improved  ATS  are  not 
available. 

A  multifaceted  "program"  must  be  initiated 
in  order  to  effectively  solve  the  ATS 
situation  without  having  to  restrict 
customer  requirements.  Standardization 
of  common  spacecraft  components  and 
subsystems,  improvements  in  ATS  project 
management  techniques,  emphasis  on  an 
effective  education  and  training  program 
for  ATS  professionals,  and  improvements  in 
ATS  design  and  performance  are  but  a  small 
part  of  a  larger  effort.  A  more  definitive 
"program"  must  be  defined  in  a  customer/ 

ATS  industry/contractor  study  that  speci- 
specifically  addresses  ATS  in  the  space 
business. 


WHAT  IS  ATS? 

•  MIL-STD-1309:  AUTOMATIC  TEST  EQUIPMENT  (ATE) 

•  MIL-STD-415D:  AUTOMATIC  TEST  EQUIPMENT  (ATE) 

-  AUTOMATIC  MONITORING  EQUIPMENT  (AME) 

-  AUTCMATIC  CHECKOUT  EQUIPMENT  (ACE) 

•  AUTOMATIC  TEST  SYSTEMS  (ATS) :  A  FULLY 

INTEGRATED  AUTOMATED  TEST  SYSTEM  THAT . 


BACKGROUND 

•  PRIMARY  PROBLEMS 

-  COMPLEXITY:  INTEGRATION  LEVEL 

-  SAFETY:  SYSTEM,  UUT,  AND  PERSONNEL 

-  TECHNOLOGY:  DIGITAL  APPLICATIONS 


.'.1 
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COMPLEXITY 


o  CURRENT  ATS  HAS: 

-  INCREASED  TESTING  THROUGHPUT  BY  USING  ITS  SPEED.  DATA  MANAGEMENT,  SELF-TEST,  AND  STANDALONE 
OPERATION  CAPABILITIES. 

INCREASED  TEST  FLEXIBILITY  BY  PROGRAMMABILITY,  INTERFACING  FLEXIBILITY,  FLEXIBLE  DISPLAY 
AND  CONTROL  CAPABILITIES. 

o  IMPACT  ON  TEST  EFFECTIVENESS: 

-  ALLOWS  FOR  FLEXIBLE  AND  INCREASED  VARIETY  OF  TESTS  TO  BE  PERFORMED  WITH  MORE  VARIETIES 
OF  STIMULI  AND  BETTER  DATA  PROCESSING. 

-  AVOIDS  MISSING  ERROR  CONDITIONS. 

o  CURRENT  ATS  HAS  SUFFERED  PROBLEMS  IN: 

-  LONG  TEST  DEVELOPMENT  TIME  (DUE  TO  UNFAMILIARITY  WITH  USE  OF  ATS,  MISUNDERSTANDING  OF  ATS 
PERFORMANCE  CAPABILITIES,  AND  NEED  TO  UNDERSTAND/USE  SIGNAL  PROCESSING  TECHNIQUES)  LEADS 
TO  LEAVING  OUT  MANY  CAPABILITIES  IN  ORDER  TO  MEET  SCHEDULE  AND ‘COST  COALS. 

-  ATS  CAPABILITIES  ARE  NOT  SUFFICIENT  FOR  MULTI-PROGRAM  USE  (DESIGNED  FOR  "TARGET"  PROGRAM 
ONLY) . 

-  INTERFACE  BETWEEN  UUT  AND  ATS  ARE  TOO  COMPLEX;  THUS  DEGRADING  PERFORMANCE  AND  INCREASING 
COSTS. 

-  INADEQUATE  SYSTEM  RESPONSE. 

SAFETY 

o  CURRENT  ATS  HAS: 

PROVIDED  TEST  REPEATABILITY  AND  ACCURACY. 

PROVIDED  FOR  SELF-TESTING. 

PROVIDED  FOR  UP-TO-DATE  TEST  STATUS  OF  ATS,  UUT,  AND  OPERATOR  ACTIONS, 
PROVIDED  HANDS-OFF  TESTING  OF  UUT  UNDER  NORMAL  TEST  OPERATIONS. 

o  IMPACT  ON  TEST  EFFECTIVENESS 
IMPROVED  TEST  CONTROL 
IMPROVED  DIAGNOSTIC  CAPABILITIES 
_  IMPROVED  ASSURANCE  OF  TEST  EQUIPMENT  PERFORMANCE. 

o  CURRENT  ATS  HAS  SUFFERED  PROBLEMS  IN: 

INADEQUATE  MAN-MACHINE  INTERFACE  ("NEED  THE  PROGRAMMER  TO  OPERATE  IT") 

INADEQUATE  UNDERSTANDING  BY  TEST  OPERATIONS  ON  WHAT  ACTIONS  THE  ATS  HAS 
TAKEN  TO  CONDUCT  A  TEST. 

ELECTROMAGNETIC  RADIATION  THAT  AFFECTS  OTHER  TEST  EQUIPMENT  AND  SECURITY 
REQUIREMENTS. 

SYSTEM  RELIABILITY. 
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TECHNOLOGY 


o  CURRENT  ATS  HAS: 

PROVIDED  COMPATIBILITY  WITH  GENERAL  TREND  TOWARD  MORE  USE  OF  DIGITAL 
COMPONENTS . 

_  PROVIDED  ANALOG  AND  DIGITAL  PROCESSING  IN  THE  SAME  SYSTEM. 

_  ALLOWED  FOR  EASIER  ATS  UPGRADING. 

PROVIDED  FOR  INTEGRATION  OF  CONTROL,  COMMANDING,  AND  DATA  ACQUISITION. 


o  IMPACT  ON  TEST  EFFECTIVENESS: 


PROVIDES  DATA  COMPATIBILITY  BETWEEN  UUT  AND  ATS. 


EASIER  TEST  INSTRUMENT  UPGRADING 


o  CURRENT  ATS  HAS  SUFFERED  PROBLEMS  IN: 

KEEPING  UP  WITH  CURRENT  ELECTRONIC  CIRCUIT  DENSITIES.  STATISTICAL  TESTING 
MAY  BECOME  PREVALENT  IN  ORDER  TO  REDUCE  TESTING  COSTS  AND  TIME. 


STANDARDIZATION 


•  PROBLEM  AREAS 


COMMERCIAL,  INDUSTRIAL,  GOVERNMENT  (INCL  MILITARY)  RELATIONSHIP: 
MUST/WILL  HAVE  TO  IMPROVE  DUE  TO  NECESSITY  (COST  AND  AVAILABILITY) 
AND  PERFORMANCE  REQUIREMENTS 


GOVERNMENT  (INCL  MILITARY)  MUST  IMPROVE  ON  SHARING  ATS  AS  MUCH  AS 
POSSIBLE 


•  POSSIBLE  ALTERNATIVES 


.  A  SIMILAR  STUDY  TO  THE  INDUSTRY /JOINT  SERVICES  AUTOMATIC  TEST  STUDY 
RECENTLY  PRESENTED  TO  THE  JOINT  LOGISTICS  COMMANDERS  MUST  BE  CON¬ 
DUCTED  FOR  SPACE  APPLICATIONS 


•  GUIDELINES 
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PROJECT/ SYSTEMS  MANAGEMEOT  APPROACH 


•  RELATIONSHIP  TO  PROJECT  MANAGEMENT  ORGANIZATION:  TEAM  CONCEPT 


METHODOLOGY 

-  SYSTEMS  APPROACH: 

TECHNICAL  -  REQUIREMENTS  ANALYSIS;  SYSTEMS  DEFINITION; 

SYSTEMS  AND  SUBSYSTEMS  DESIGN;  DEVELOPMENT; 
TEST,  VERIFICATION,  AND  VALIDATION;  OPERATIONS 
AND  MAINTENANCE;  END-OF-USE, 

NONTECHNICAL  -  PROJECT  CONTROL,  MATRIX  ORGANIZATION, 


»  GUIDE  LINES 


-  PARTITIONED  RESOURCES  APPROACH 


PIECE-MEAL  APPROACH 


TRIAL-AND-ERROR  APPROACH 


MAJOR  ATS  PROJECT  IMPACTS 


•  CUSTOMER 

-  DEVELOPMENT  AND  USE  OF  ATS  MUST  INCREASE  SIGNIFICANTLY 

-  ATS  DEVELOPMENT  AND  USE  WILL  APPROACH  PROJECT  SIGNIFICANCE.  IT  WILL  NOT  REMAIN  SIMPLY  AS  AN 
ADJUNCT  TO  A  PROGRAM. 

-  MUST/NEED  TO  INITIATE  AN  OVERALL  "INDUSTRY"  STUDY  ON  ATS. 

•  MANAGEMENT 

-  PROJECT-ORIENTED:  QUALIFIED  ATS-ORIENTED  DEVELOPMENT  AND  OPERATIONS  TEAM 

-  USE  OF  SYSTEMS  APPROACH 

-  NEED  TO  GET  ACTIVELY  INVOLVED  IN  MAKING  SURE  THAT  THERE  IS  AN  ADEQUATE  SUPPLY  OF  QUALIFIED 
ATS  PERSONNEL: 

•  ATS  DEFINITION 

-  MODULAR  HARDWARE  AND  SOFTWARE  THAT  PERFORMS  AS  A  HIGHLY  INTEGRATED  SYSTEM. 

-  STANDARDIZED  INTERFACES 

-  HUMAN  ENGINEERED  FOR  OPERATIONS  AND  MAINTENANCE 

-  REALTIME,  CLOSED  LOOP  OPERATION 

-  DISTRIBUTED  PROCESSING  WITH  CENTRAL  CONTROL 

-  PORTABLE 

-  USE  OF  HIGH  LEVEL  TEST  LANGUAGE 

-  AUTOMATED  TEST  PROCEDURE  GENERATION 

-  EXTENSIVE  USE  OF  SYSTEM  ARCHITECTURE  ANALYSIS 

-  DOCUMENTATION. 


TEST  ACCOMODATION  IN  DESIGN  (SYNOPSIS) 

J.A.  Durschinger 
TRW/DSSG 

Satellite  design  features  to  facilitate 
system  testing  needs  to  be  an  integral 
part  of  the  design  process.  Hardware  and 
design  realities  must  be  recognized  - 
there  are  limitations,  opportunities,  and 
"that  is  how  it  is"  situations.  Many 
tradeoffs  and  associ2\ted  "payoffs"  must  be 
considered.  Some  specific  recommendations 
are  offered  for  both  test  and  design 
engineering  people. 


TRADE  OFFS  ARE  REQUIRED  TO  DEFINE 
TEST  ACCOMMODATION  REQUIREMENTS 

•  HARDLINE  CONNECTIONS  VS  TELEMETRY  MEASUREMENTS 

•  DATA  BUS  INTERFACES  VS  COMMAND/TELEMETRY  INTERFACES 

•  ONE  TIME  INSTALLATION  VS  REHOVAL/REPLACEMENT/REUSE 

•  TEST  TIME  COSTS  VS  EGSE  COST 

•  TEST  INSTRUMENTATION  VS  TELEMETRY  MEASUREMENTS 

•  STIMULATION  VS  SIMULATION 

•  MISSION  PROFILE  TESTING  VS  FUNCTION  TESTING 

•  LAUNCH  SITE  TESTING  VS  SHIP  AND  SHOOT  CONCEPT 


«HAI  IS  TEST  ACCimiDATlONS? 


•  IT  IS  SATELLITE  DESIGN  TO  FACILITATE  SYSTEM  TEST. 

*  IT  IS  A  FACTOR  IN  THE  SATELLITE  DESIGN  PROCESS  WHICH  INTEGRATES  TEST 
REOUIREMENTS  INTO  THE  SYSTEM  DESIGN. 


INCORPORATING  TEST  ACCOMIIODAf IONS  INTO  SATELLITE  DESIGN 
RECOGNIZES  HARDWARE  AND  DESIGN  REALITIES 

•  UNITS  DO  FAIL 

•  DESIGN  niSTAKES  DO  OCCUR 

•  DESIGN  CHANGES  DO  OCCUR 

•  GENERIC  DEFECTS  ARE  DISCOVERED 

•  SOME  TYPES  OF  EOUIPMENT  REQUIRE  SPECIAL  STORAGE 

«  SOME  TYPES  OF  EOUIPMENT  MUST  BE  INSTALLED  LATE  DURING  PRE-LAUNCH  OPERATIONS 

•  TOTAL  SYSTEM  HARDWARE  MUST  BE  TESTED,  NOT  JUST  SYSTEM  FUNCTIONS 
t  PERFORMANCE  BUKETS  DO  NOT  ALLOW  FLYING  ALL  TEST  INSTRUMENTATION 


WHAT  IS  THE  PAYOFF  FOR  TEST  ACCOMMODATION  CONSIDERATIONS 

•  INCREASED  ASSEMBIY  AND  TEST  EFFICIENCY. 

•  ENHANCED  HARDWARE  AND  PERSONNEL  SAFETY. 

•  INCREASED  THROUGHTNESS  AND  COMPLETENESS  OF  SYSTEM  TESTING. 

•  FACILITATED  DISCOVERY  OF  SUBTLE  SYSTEM  PROBLEMS. 

SOME  SPECIFIC  TEST  ACCOMP'iODATlOH  RECOMMENDATIONS  TO  OUR  ENGINEERING  PEOPLE 

•  ESTABLISHING  A  DEDICATED  COMMUNICATION  FORUM  TO  WORK  TEST  ACCOMMODATION 
ISSUES  BETWEEN  TEST  AND  ENGINEERING. 

•  DEFINING  QUAIITITATIVE  CLEARANCE  REQUIREMENTS  FOR  SATELLITE  HARNESS 
CONNECTORS. 

t  MOUNTING  UNITS  REQUIRING  ALIGNMENTS  ON  THREE  POINTS  AND  USING  ADJUSTMENT 
SCREWS  FOR  ALIGNMENT. 

•  REQUIRING  ALL  ELECTRONIC  UNITS  CONNECTED  TO  THE  SATELLITE  MAIN  POWER  BUS 
TO  BE  CAPABLE  OF  RAMP  POWER  TURN-ON. 

•  TO  THE  MAXIMUM  EXTENT  POSSIBLE  AHACH  ALL  THERMAL  BLANKETS  WITH 
MECHANICAL  FASTENERS,  NOT  TAPE. 

•  DEFINING  STANDARDIZED  TEST  POINTS  AND  IN  FLIGHT  JUMPER  CONFIGURATIONS 
FOR  BATTERIES,  SOLAR  ARRAYS,  AND  POWER  DISTRIBUTION  SYSTEMS. 


!• 
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EVALUATION  OF  TEST  EFFECTIVENESS  RESULTS 

Dr.  A.  Krausz 
Chief  Test  Engineer 
Assembly  and  Test  Operations 
Space  Systems  Division,  TRW  DSSG 

This  presentation  will  address  three  basic 
questions;  What  is  test  effectiveness? 

How  can  test  effectiveness  be  measured? 

What  is  the  test  effectiveness  of  recent, 
spacecraft  projects?  These  questions 
can  be  answered  by  focusing  on  the  design 
and  workmanship  defects  which  must  be  found 
by  testing  and  by  considering  the  monetary 
and  schedule  cost  of  the  test  program. 

Data  obtained  from  quality  assurance 
records  for  several  recent  spacecraft 
projects  will  be  presented  which  show  the 
number  of  defects  found  during  various 
phases  of  the  testing  program  and  during 
orbital  operation.  The  data  indicate  that 
rough  correlation  exists  between  the  number 
of  orbital  anomalies  and  the  normalized 
number  of  factory  test  discrepancies. 
Although  on-orbit  operation  of  satellites 
has  been  excellent,  some  suggestions  for 
improving  the  test  programs  are  made. 


KEASURCKCNT  OF  TEST  EFFECTIVENESS 


a  EASED  ON  TEST  DISCREPANCIES  (DEFECTS) 


a  DEFINITIONS 


CO  TEST  EFFECTIVENESS 


NUMBER  OF  DEFECTS  FOUND  BY  TEST 
TOTAL  NUMBER  OF  DEFECTS  PRESENT 


fe>  ESCAPE  RATIO  = 


•  ASS'Ji'.PIlOriS 


NUMBER  OF  DEFECTS  NOT  FOUND 
TOTAL  NUF.BER  OF  DEFECTS  PRESENT 


•ft  DEFECTS  EXIST  PRIOR  TO  TEST 


0*  TESTS  CAN  LOCATE  DEFECTS 


IS  TEST  EFFECTIVENESS? 


SATISFACTION  CF  TEST  OSJECTjVES 


C.'  ALL  0?J£CTIVES 


LOW  COST 


CC  MINIMUM  TEST  TIME 


0  TEST  OBJECTIVES 


QUESTIONS 


C:  LOCATE  AND  ELIMINATE  DEFECTS 


CO  DEMONSTRATE  CONFORMANCE  TO  REQUIREMENTS 


?  WiiAT  IS  min  BY  TEST  EFFECTIVENESS? 


-  HOW  CAN  TEST  EFFECTIVENESS  BE  MEASURED? 


r  v:::at  is  the  test  effectiveness  of 

CL’RREMT  spacecraft  PROJECTS? 


PROJECT  A  -  CAUSE  OF  TEST  DISCREPANCIES 


UEFECT/CAUSE  OF  DISCREPANCY 


p;'n:;:,CT  ncrrcrs 


COIIPONENT 


_ TEST  LEVFI _ _ 

SPACECRAFT j LAUNOI  BASE  |  ON  ORBIT 


i  r  t,- !LU'  E  68 

:  ' 1  r;.;-.MjrACTURlNG  112 

r/ :  m,-;; ULMGN  174 

E'  [  i-  '.rt  ASS' ::cLY 
.  '.i  I.E'  !  I  EtSIGN 

:  A  ; 'I  13 

e:,;.:  :i -EiiiLi) 


:  :'l:,!at;cn 

:l  iflL'll'I'LNT 
.  ■  I  -'ArOE  I.EROR 
:  E-.;8P 


M'N-'TTECTS 


;  UiTiiri  TOEEPANCE/WAIVER  78 

I  UNAJLE  TO  REPEAT  PRCSLEM  29 


NOTES:  4  spacecraft,  3  in  orbit,  3  years  of  orbital  perforinance 


PROJECT  B  -  CAUSE  OF  TEST  DISCREPANCIES 


2EFLCT/CACSE  OF  DISCREPAfiCY 


piA^c'JCTjiErrcis 

I’ART  !  a: LURE 

C';:’P''r:r';T  manufacturing 
>  ■yi'.:  :;i  ursiCN 
S:'.'CLf'  ■■ri  ASSE"CLY 

S!''a::ipaft  oesign 
tlst  iNPruEi) 

I  ••CLTEM'INEl) 


PRCCEJ’JRE  DEFECTS 

DOCUMENTATION 
TEST  EQUIPMENT 
CPERATCR  ERROR 
TEST  SETUP 


NON-DEFECTS 

WITHIN  TOLERANCE/WAIVER 
UNABLE  TO  REPEAT  PROBLEM 


TEST  LEVEL 


COMPONENT 


SPACECRAFT 


NOTES;  3  spacecraft,  3  on  orbit,  4  years  cf  orbital  perfori;!ance 
Experiment  TDRs  not  included 
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PROJECT  A  -  LOCATION  OF  TEST  DISCREPANCIES 


ELECTRONIC  COMPONENTS 

PROPULSION/STRUCTURE 

HARNESS 

TEST/DEFECT 

Part 

Workman¬ 

ship 

Design 

Part 

Design 

Workman¬ 

ship 

Design 

TOTAL 

DEFECTS 

UNIT  ACCEPTANCE 

68 

100 

167 

1 

12 

7 

354 

SPACECRAFT 

ACCEPTANCE 

•11 

15 

25 

2 

23 

2 

7 

2 

87 

Integration 

(3) 

(6) 

(12) 

(1) 

(11) 

(1) 

(3) 

(1) 

(38) 

First  Functional 

(3) 

(2) 

(5) 

(10) 

Temperature 

(2) 

(21 

(7) 

(1) 

(2) 

(1) 

(15) 

EMC/IM 

(1) 

(1) 

(1) 

(4) 

(1) 

(3) 

Dynamics 

Post  Dynamics 

(4) 

(1) 

(5) 

T/V  Test 

(2) 

(2) 

(4) 

Preship 

Functional 

(2) 

(3) 

(1) 

(6) 

Launch  Base 

(1) 

(1) 

ON  ORBIT 

1 

1 

c 

4 

PROJECT  B  -  LOCATION  OF  TEST  DISCREPANCIES 


ELECTRONIC  COMPONENTS 

PROPULSION/STRUCTURE 

HARNESS 

TEST/DEFECT 

Part 

Workman¬ 

ship 

Design 

Part 

Workman¬ 

ship 

Design 

Workman¬ 

ship 

Design 

TOTAL 

DEFECTS 

UNIT  ACCEPTANCE 

35 

79 

43 

3 

2 

162 

SPACECRAFT 

ACCEPTANCE 

5 

6 

23 

7 

1 

11 

12 

65 

Integration 

(9) 

(5) 

(1) 

(2) 

(9) 

(26) 

EMC 

(2) 

(1) 

(1) 

(1) 

(5) 

First  Functional 

(1) 

(2) 

(4) 

(1) 

(3) 

(11) 

Dynamic 

Environment 

(1) 

(1) 

(1) 

(1) 

(4) 

Post  Dynamic 
Functional 

(1) 

(2) 

(2) 

(5) 

T/V  Environment 

(1) 

(1) 

Preship  Functional 

(1) 

'D 

(1) 

(2) 

(5) 

ETR 

(1) 

(1) 

(4) 

(2) 

(8) 

ON  ORBIT 

1 

5 

1 

7 

0:i  ORBIT  DISCREPA^;CIES  PER  1000  PARTS 


PRELIMINARY  CONCLUSIONS 

0  CURRENT  TEST  PROGRAMS  YIELD  SUCCESSFUL  ON-ORBIT  OPERATION. 
UNDETECTED  DESIGN  DISCREPANCIES  ARE  THE  PRINCIPAL  CAUSE 
OF  ANOMALIES  WHICH  DO  OCCUR. 

0  A  LARGE  NUMBER  OF  COMPONENT  DEFECTS  ESCAPE  DETECTION  DURING 
COMPONENT.  ACCEPTANCE  TESTING  AND  ARE  FOUND  BY  SYSTEMS  TESTS. 
ABOUT  60%  OF  SYSTEMS  TEST  DEFECTS  ARE  SUCH  "ESCAPES". 

0  ABOUT  2/3  OF  SPACECRAFT  DEFECTS  RESULT  FROM  INTEGRATION  AND 
INITIAL  ROOM  AMBIENT  FUNCTIONAL  TESTING. 

0  TEMPERATURE  CYCLING  OF  THE  SPACECRAFT  IS  MORE  PRODUCTIVE 
THAN  EXTENDED  THERMAL  VACUUM  TESTING. 


ORBITAL  PERFORMANCE  CORRELATION  BY  PROJECT 

(Defect  rates  vary  from  project  to  project,  but  correlation 
between  orbital  performance  and  test  results  holds) 

Project  A  -  3  Military  Comsats 
Project  B  -  3  Scientific  Spacecraft 
Project  C  -  7  Classified  Spacecraft 
Project  0  -\12  Commercial  Comsats 


TOTAL  TEST  DEFECTS  PER  1000  PARTS 


.1 


K- 

B 


.2  .3  .4  .D 

SPACECRAFT  TEST  DEFECTS  PER  1000  PARTS 


.7 


SOME  RECOMMENDATIONS 

9  DESIGN  TEST  PROGRAMS  TO  FOCUS  ON  POSSIBLE  DESIGN  DISCREPANCIES 
BY  CONDUCTING  MORE  REALISTIC  END-TO-END  PERFORMANCE  TESTS. 

0  CORRECT  DEFECTS  AS  EARLY  AS  POSSIBLE  SINCE  COST  IMPACT  OF 
FAILURES  AT  THE  SPACECRAFT  TEST  LEVEL  IS  USUALLY  AT  LEAST 
5  TIMES  AS  GREAT  AS  AT  COMPONENT  LEVEL. 

9  INCREASE  THE  COMPONENT  LEVEL  TEST  DURATION  AND  ENVIRONMENTAL 
STRESS  LEVELS  TO  DISCOVER  INCIPIENT  FAILURES  PRIOR  TO 
SPACECRAFT  INTEGRATION. 

0  PERFORM  TEMPERATURE  CYCLING  TESTS  RATHER  THAN  LONG  DURATION 
T/V  tests  at  the  highest  POSSIBLE  LEVEL  OF  ASSEMBLY  SINCE 
THIS  IS  MORE  COST  EFFECTIVE  IN  FINDING  DESIGN  AND  WORKMANSHIP 
DEFECTS, 


•  ... 


.  '*•  ^ •>  •’* 


209 


TEST  SORWARE  (SYNOPSIS) 

E.  W.  Dusio 

Manager,  Software  Engineering 
RCA  Astro-Electronics 

This  discussion  presents  several  different 
approaches  to  test  software  development 
and  usage  at  RCA  Astro-Electronics  and 
compares  them  with  respect  to  (1)  ease  of 
developing  the  test  programs,  (2)  ease  of 
debugging  the  units  under  test,  and 
(3)  ease  and  throughness  of  testing. 
SoHware  development  facilities  and  self¬ 
testing  software  for  flight  computers  are 
addressed.  Some  possible  topics  for 
further  workshop  discussion  are  also 
identified. 


TEST  SOFTWARE 


•  TEST  SOFTWARE  LANGUAGES , USED  AS  ASTRO 

•  LANGUAGE  CHARACTERISTICS 


•  TEST  SOFTWARE  DEVELOPMENT  FACILITIES 


•  SELF  TESTING  SOFTWARE  FOR  FLIGHT 
COMPUTERS 


•  POSSIBLE  TOPICS  FOR  DISCUSSION 


TEST  SOFTWARE 
LANGUAGES  USED  AT  ASTRO 

•  ATLAS:  A  SUBSET  OF  STANDARD  ATLAS 


•  BASIC:  HEWLETT  PACKARD'S  BASIC 


•  TEST  CONTROL  LANGUAGES:  SEVERAL  HOME 
BREWED  VERSIONS 


LANGUAGE  CHARACTERISTICS 

•  .TlAS:  S0.SC[  C0-'l>IL[il  10  I  »  I  [  ant  D  I  A  T  E  CODE  PRIOR  TO  ERECUTIOK 

•  OASIC:  Li:.E-B»-tr.t  i  S  TE  RPRE  T  E  «  ElECOTCS  SOURCE. 

•  TCL'S:  DOTH  cohreler  and  enterpreter  versions  have  seen  built. 


ATLAS  AND  BASIC 
LANGUAGE  CHARACTERISTICS 

•  "stanoaud"  languages 

•  limited  transportaoilitv  to  other  svstems 

•  VERV  fLEAISLE,  CAN  EXERCISE  ALL  TEST  EOUIPMENT  OPTIONS 

•  TEST  WRITERS  REQUIRE  NITTV  GRITTY  KNOWLEDGE  OF  TEST  STATION 


TEST  CONTROL  LANGUAGES 
LANGUAGE  CHARACTERISTICS 


•  INTENTIONALLY  LIMITED  FLEXIBILITY 

•  TEST  STEP  ALGORITHMS  ARE  LOCKED  IN  THE  SOFTWARE 

•  A  TEST  =  A  SEQUENCE  OF  TEST  STEPS 


TEST  SOFTWARE  DEVELOPMENT  FACILITIES 

•  SEPARATE  DEVELOPMENT  FACILITIES  WITHOUT  TEST  BUS 
ANO  EQUIPMENT. 

-  ONLY  A  LIMITED  TEST  OF  PROCEDURES  IS  SUPPORTED 

•  SHARED  TEST/DEVELOPMENT  FACILITY 

-  THE  AVAILABLE  TEST  TIME  IS  REDUCED  BY  DEVELOPMENT 
TIME  ALLOCATION 

•  ULTIMATELY  THE  PROCEDURE,  TEST  STATIUII  AND  UUT  ARE 
DEBUGGED  TOGETHER 


DEBUGGING  UUTS  WITH  TEST  SOFTWARE 

atlas 

A  manual  mode  procedure  chained  into  test  proce.'-jre 

ALLOWS  KEYBOARD  CONTROL  OF  TEST 
BASIC 

INSERT  DEBUG  CMOS  FROM  KEVaARD. 

TCL 

MANUAL  MODE  BUILDS  NEW  TEST  SEQUENCES  FRO:i  KEYBOARD. 


THROUGHNESS  OF  TESTS 

TiulKOLGhNCSS  iir  T:‘,IS  IS  A  ril'.CTlUS  OF  T  HOROOGliriC  SS 
OF  3AIA  [VUUAT'.CN 

•  A  CQSTISFOoS  n.‘CK'',kl),.',0  profess  HOIIIIORS 

ilSEXPiOU:'  STATE  CloSSOES  OR  PARA‘TETER  ORIFTS  OR: 

t  THE  DATA  IS  SATIPElD  UPON  DEMAND. 

IN  BOTH  CASES  HARDNARE  FAULT  MONITORS  PROTECT  UMT  UNDER 
TEST 


SELF  TESTING  SOFTWARE 
FOR  FLIGHT  COMPUTERS 


TOPICS  FOR  DISCUSSION 

,  !  . .  AN3  SXSItM  TIST 


:■ .  hM.OUES, 


:  R  CiOFIil  . 


A'A  1  i  AljA!  ;OT. ,  A!10  It  ST 


tM 


:j+2 


TEST  STEP  ALGORITHM  UPDATE  CLOCK 

ri43  N44  r;F5 


40 


50 


A:  ddd:hh:iTim:ss 

B:  niLLISEC 


■  RA'-'E 


N4  1 

ACQUIRE 

CLOCK 

WORDS 

FOR  REFERENCE 

N4? 

COMMAND 

UPDATE 

CLOCK 

,  VERIFY  NORMAL 

N  +  3 

VERIFY 

1 

JUMP  IN 

TIME 

TO  UPDATED  VALUE 

:a5 

1 

\  VERIFY 

NORMAL 

CLOCK 

SEQUENCING 

tU6  J 

1 

6C 


fiOTE;  AT  ALL  OTHER  TIMES,  A  BACKGROUND  PROCESS  MONITORS  FRAME  TO  FRAME 
SEQUENCING  OF  TIME  CODE 


SUMMARY  OF  ISSUES 
AND  RECOMMENDATIONS 


TEST  EFFECTIVENESS 
SUMMARY  OF 

ISSUES  AND  RECOMMENDATIONS 


1.  Test  Data  Exchange 
Issue 

There  is  a  lack  of  data  interchange 
among  projects  and  contractors 
which  would  allow  better  decision 
making  regarding  the  most  produc¬ 
tive  test  procedures  at  the  various 
levels  of  assembly  and  test  envir¬ 
onments  to  maximize  test  effect¬ 
iveness  and  reduce  overall  test 
costs. 

Recommendations 

o  Establish  a  committee  including 
AF  Space  Division,  NASA  and 
contractors  to  provide  a  continu¬ 
ing  forum  for  exchange  of  test 
program  data  and  experiences  at 
the  subassembly,  component 
(boxi,  subsystem  and  system 
test  levels.  Related  orbital  per¬ 
formance  results  would  be  includ¬ 
ed.  Significant  conclusions  would 
be  shared  with  engineers  and  pro¬ 
ject  management  personnel. 
Growing  complexity  at  lower 
levels  of  assembly  makes  the 
need  more  acute.  There  is  a 
very  strongly  and  nearly  unani¬ 
mously  felt  need  for  this  exper¬ 
ience  sharing  forum  as  a  contin¬ 
uing  but  not  overly  formalized 
working  group.  The  Space  Parts 
Working  Group  was  mentioned  as 
a  possible  model. 

2.  Failure  Information  Exchange 
Issue 

Accumulation  of  failure  information 
from  systems,  subsystem  and  com¬ 
ponent  (box)  test  programs  across 
many  programs  is  extremely 
difficult.  Summary  data  is  not 
uniformly  or  consistently  required 
on  contracts.  Formats  and  con¬ 


tents  are  highly  variable.  Data 
base  accumulation  is  needed  to 
examine  and  validate  proposed 
changes  to  current  test  philoso¬ 
phies,  approaches  and  procedures. 

Recommendations 

©Examine  existing  industry  and 
government  reporting  systems 
and  develop  a  compatible  com¬ 
mon  data  requirement  (Data  Item 
Description)  for  use  in  contracts. 
Intent  would  be  to  accept  con¬ 
tractors'  formats,  so  long  as 
certain  key  information  is  in¬ 
cluded. 

3.  MIL-STD-1540A 
Issue 

Many  changes  to  the  technical  re¬ 
quirements  now  in  MIL-STD- 
1  540A  have  been  proposed  by  both 
industry  and  government.  In  most 
cases,  the  data  base  is,  as  yet, 
insufficient  to  fully  support  the  pro¬ 
posed  changes.  A  major  redraft 
of  MIL-STD-1540  is  now  in  pro¬ 
gress.  Reviews  and  working  meet¬ 
ings  are  forecast  for  September- 
October  1980,  with  subsequent 
publication  about  March  1981. 

Recommendations 

©Acquire  and  analyze  additional 
information  from  component 
(box),  .subsystem  and  system 
test  programs  to  allow  logical 
and  supportable  decisions  about 
significant  changes  in  require¬ 
ments.  Update  the  require¬ 
ments  of  MIL-STD-1540  to  in¬ 
clude  those  changes  and  im¬ 
provements  which  will  improve 
test  effectiveness,  efficiency 
and  perceptiveness. 

4.  Qualification  Flight  Items 
Issue 

Test  guidance  and  technical  re¬ 
quirements  for  "buy-one-fly-one  ", 
Protoflight  or  "fly  qualification 
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6.  Automated  Testing 


items"  is  minimally  addressed 
in  MIL-STD-1540A.  These 
approaches  are  used  for  various 
sound  reasons  by  many  programs. 
Test  baseline  guidance,  including 
technical  requirements  is  needed 
for  these  kinds  of  programs  and 
projects. 

Recommendations 

o  Implement  a  section(s)  in  the 
next  issuance  of  MIL-STD- 
1540  which  'covers  these  areas. 
A  major  redraft  of  the  stan¬ 
dard  is  now  in  progress  with  a 
publication  target  of  March 
1981. 

Testabiiity 

Issue 

Testability  of  many  existing  de¬ 
signs  is  not  adequate  for  evalua¬ 
tion  of  conformance  to  perform¬ 
ance  requirements  and  identifica¬ 
tion  of  discrepancies  through  test¬ 
ing  at  component  (box),  subsystem 
and  system  levels,  nor  for  analy¬ 
sis  of  orbital  performance  anoma¬ 
lies.  Newest  technologies  (hy¬ 
brids,  VLSI,  etc.)  compound  the 
problem.  Testability  must  be  an 
"up-front"  requirement,  demanded 
and  addressed  from  the  earliest 
program  phases  onward. 

Recommendations 

o  The  customer  should  require 
adequate  attention  to  testability 
approaches,  philosophies,  and 
design  features  from  program 
inception  onward.  Testability 
design  features  should  be  address 
ed  in  the  RFP  and  proposals, 
with  subsequent  review  and  appro¬ 
val  of  their  implementation 
throughout  the  development  cycle. 
There  is  no  "Royal  Road"  to  test¬ 
ability,  but  rather  there  are  a 
number  of  approaches  and  tech¬ 
niques  which  are  appropriate  for 
various  technologies  and  program 
trade-offs. 


Issue 

Automated  test  requirements 
forecast  to  be  increasing  at  an 
exponential  rate  due  to  increas¬ 
ing  system  complexity  and  tech¬ 
nology  changes.  Automatic  test¬ 
ing  techniques,  configurations, 
software  languages,  etc.  have 
also  been  "exploding".  A  forum 
for  customers,  contractors  and 
Automatic  Test  Equipment  manu¬ 
facturers  is  needed  which  will  re¬ 
cognize  the  concerns  and  needs  of 
the  space  systems  community. 

Recommendations 

oAn  effort  similar  to  the  industry/ 
joint  services  automatic  test  pro¬ 
ject  should  be  accomplished, 
with  a  subsequent  continuing  foru 
forum  to  be  established.  The  con 
concerns  for  automated  testing  of 
space  systems  are  not  necessar¬ 
ily  different  from  those  of  other 
services  or  industry  segments, 
but  do  appear  to  have  different 
urgency  levels  for  some  pro¬ 
blems,  due  largely  to  the  few-of- 
a-kind  but  extremely  complex 
nature  of  the  systems  involved. 
Governmental  or  major  industry 
association  sponsorship  is  seen 
as  necessary. 

7.  Testability  in  Design 
Is  sue 

Many  automated  test  systems 
(ATS)  are  developed  under  ex  - 
treme  cost,  schedule  and  resource 
const  ra ints -  to  meet  program  needs 
and  limitations.  This  is  true  both 
for  program  unique  ATS  and  for 
ha  rdwa  re/softwa  re  interfacing  to 
common  ATS.  Growing  test  re¬ 
quirements  magnify  the  problems 
and  program  risks  in  cost,  sche¬ 
dule  and  resources.  ATS  is  or 
will  be  a  major  program  conce  rn 
and  should  be  addressed  as  such. 


Recommendations 


oAutomated  test  systems  per¬ 
sonnel  must  be  involved  as  early 
as  possible  in  concept,  RFP, 
proposal  and  design  development. 
Deferral  assures  later  problems 
and  detrimental  compromises. 

o  Project/Systems  management 
approaches  need  to  be  more  wide¬ 
ly  implemented  in  the  ATS  arena 
to  assure  better  program-ATS 
compatibility  and  more  effective 
and  efficient  ATS  acquisitions. 

o  The  automated  testing  area  needs 
greater  emphasis  in  RFP,  SOW, 
proposals  and  contract  manage¬ 
ment  tool  (WBS  for  example). 

Test  Language 

Issue 


The  lack  of  adequate  high  level 
test  languages  is  adversely  im¬ 
pacting  test  costs,  schedules, 
resources  and  effectiveness.  As 
spacecraft  complexity  increases, 
and  the  impact  of  not  having  ade¬ 
quate  computer  languages  for  auto¬ 
mated  tests  is  further  magnified. 

Recommendations 


o  Investigate,  define  and  develop 
a  high  level  test  language  for  use 
on  automated  test  systems. 

NASAs  "STOL"  system  test  lan¬ 
guage  should  be  examined  for 
system  test  applications. 

Current  language  standardization 
efforts  do  not  appear  to  offer  any 
relief  for  the  spacecraft  test 
language  problem,  at  least  not  in 
the  reasonably  near  term. 
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USAF  SPACE  DIVISION 
SPACE  PARTS  ACTIVITIES 

Lt  Col  Ken  Blakney 

USAF  Space  Division 

Space  Division  (SD)  is  cotnnitted  to 
improving  the  reliability  and  availability 
of  all  the  parts  used  in  the  spacecraft 
and  launch  vehicles  managed  by  SD.  This 
short  presentation  will  summarize  (1)  the 
reasons  for  that  commitment,  as  well  as 
(2)  Space  Division  activities  (past, 
present,  and  planned)  that  are  aimed  at 
developing  standard  space  quality  parts, 
requirements,  techniques,  documents,  and 
sources  and  making  space  quality  parts 
available  for  use  by  all  space  programs. 

I  will  also  briefly  discuss  the  problems 
and  challenges  we  face.  We  hope  that  the 
discussions  of  this  workshop  will  bring 
these  challenges  into  better  focus  and 
formulate  some  implementable  recommended 
solutions  to  those  challenges  which  we 
have  not  already  solved. 

In  trying  to  clarify  the  Issues  and  formu¬ 
late  solutions  to  the  "parts  problems" 
confronting  us,  we  must  keep  in  mind  the 
unique  features  of  the  systems  (space¬ 
craft  and  launch  vehicles)  and  situation 
we  are  addressing. 

a.  Spacecraft  and  launch  vehicle 
contract  "buys"  generally  consist  of  less 
than  ten  systems  or  subsystems  and  entire 
spacecraft  programs  often  Involve  only 
two,  three,  or  four  spacecraft.  Thus, 
statistical  sampling  and  learning  curves 
are  not  very  useful  for  achieving  space¬ 
craft  reliability.  We  must.  Instead, 
rely  on  "up  front"  engineering,  failure 
prevention,  and  doing  it  right  the  first 
time. 

b.  Because  of  the  evolution  of  pro¬ 
gram  mission  requirements,  rapidly 
changing  technologies,  and  the  autonomous 
nature  of  each  program/prime  contractor 
combination,  each  spacecraft  "set" 
generally  is,  at  least  partially,  unique 
in  its  design  features.  Once  again,  the 
learning  curve  and  reliability  growth 
have  limited  applicability.  We  must, 
instead,  maximize  the  use  of  lessons 
learned  and  standard  parts,  techniques, 
and  documents  in  order  to  translate  the 
successful  practices  from  one  program  to 
another. 


c.  Because  we  are  unable  to  remove 
and  replace  field  items  in  orbit,  we 
must  design  and  fabricate  our  spacecraft 
from  the  most  reliable  parts,  materials, 
processes,  and  design  features  available 
to  prevent  premature  failure  of  the 
system. 

d.  "Replacement"  of  a  failed 
spacecraft  or  launch  vehicle  involves 
multi-million  dollar  funding  and  a 
significant  percentage  of  program  dollars, 
schedules,  and  equipment  assets.  Our 
approach,  therefore,  must  be  to  take  no 
shortcuts  that  would  increase  the  risk 

of  premature  system  failure. 

Spacecraft  reliability  requirements 
have  become  increasingly  difficult  to 
achieve  because  of  (1)  increased  life¬ 
time  requirements  and  (2)  increased 
system  complexity  and  part  count.  As 
shown  below,  SD  spacecraft  have  evolved 
from  required  Mean  Mission  Durations 
(average  expected  time  to  failure)  of 
12  and  18  months  to  required  Mean  Mission 
Durations  of  7  years  (84  months).  At 
the  same  time,  these  systems  have 
increased  in  complexity  more  than  15 
fold.  It  is  encouraging  to  note  that 
selected  SD  spacecraft  have  achieved 
successful  operation  on  orbit  of  5b, 

6+,  and  9  years.  To  date,  however,  the 
average  time  to  failure  of  SD  spacecraft 
is  less  than  5  years.  If  we  are  to 
achieve  the  desired  MMD  of  7  years,  we 
must,  therefore,  improve  the  reliability 
of  the  parts  used  to  design  and  build 
our  current  and  future  spacecraft. 


MMD  Required 

Electronic 

(Months) 

Parts 

IDCSP 

18 

4,700 

DMSP,  Block 

4 

12 

10,000 

DMSP,  Block 

5A/B/C  5.2 

23,600 

NATO  II 

36 

12,900 

DMSP,  Block 

5D 

19.7 

25,000 

DSCS  II 

36 

36,000 

NATO  III 

56 

21,000 

FLTSATCOM 

37 

60,000 

GPS,  PHASE 

I 

48 

33,500 

DSCS  III 

84 

150,000 

GPS.  PHASE 

III 

72 

TBD 

STRATSAT 

84 

TBD 

No.  MMD  Achieved  Longest 
Op.*  (Months)  Llfe(^nths) 


IDCSP 

0 

73 

108 

DMSP, 

Block  4 

0 

13.7 

25.6 

DMSP. 

Block  5A/B/C  0 

16.1 

39.8 

NATO  II 

0 

45.5 

66 

DMSP, 

Block  5D 

1 

20.7 

31.2 

DSCS  II 

7 

22.7 

76  + 

NATO  III 

3 

36 

48  + 

FLTSATCOM 

3 

12.1 

24.4+ 

GPS,  PHASE 

I  5 

15.9 

24.5+ 

DSCS  III 

TBD 

GPS.  PHASE 

III 

TBD 

STRATSAT 

TBD 

♦Satellites 

still 

operating  may  increase 

MMD 

+This  particular  satellite  still  operating 

Space  Division's  launch  vehicle  relia¬ 
bility  record  is  very  good.  The  success 
ratio  of  over  280  launches  over  the  past 
15  years  is  95%.  We  require,  however, 
that  our  new  launch  vehicles  achieve  96% 
to  97%  reliability.  And  due  to  the  enor¬ 
mous  cost  of  replacement  of  launch  vehi¬ 
cles  and  their  payloads,  we  desire  and 
strive  for  100%  reliability. 

These  system  requirements  and  constraints 
are  reflected  into  the  parts  level  require¬ 
ments  and  constraints.  Since  a  space¬ 
craft  is  composed  of  more  than  100,000 
parts,  mostly  electronic,  and  since  a 
single  part  can  cause  failure  of  the 
entire  spacecraft  or  mission,  every  part 
must  be  of  the  highest  reliability  attaina¬ 
ble.  Redundancy  can  help  alleviate  this 
situation,  but  due  to  system  weight 
constraints,  redundancy  is  limited  and  is 
usually  not  practiced  at  the  part  level. 
Also,  redundant  unreliable  parts  will 
certainly  not  allow  achievement  of  7  year 
spacecraft  lifetimes. 

Since  the  1978  Mission  Assurance  Conference, 
we  have  made  considerable  progress  toward 
development  of  standard  "space  quality" 
requirements,  specifications,  test  methods, 
and  other  Improvements  in  the  reliability 
of  parts  used  in  our  space  systems.  Space 
Division  programs  are  still,  however, 
experiencing  difficulties  in  procuring 
parts  of  sufficient  reliability  to  support 


program  requirements.  Small  quantity 
parts  buys,  divergence  of  space  require¬ 
ments  from  available  "commercial"  parts, 
rapidly  changing  LSIC  technology,  and 
resultant  part  manufacturer  lack  of  inter¬ 
est  in  small  quantity,  "special"  space 
parts  are  some  of  the  underlying  causes 
of  the  parts  procurement  difficulties. 

The  ultimate  results  of  our  inability  to 
procure  readily  available  "space  quality" 
parts  include  program  schedule  slips,  re¬ 
designs  and  part  retrofits,  cost  overruns, 
and  compromises  in  the  part  and  program 
reliability. 

The  following  list  sunmarizes  the  high¬ 
lights  of  the  parts  panel  conducted  at 
the  1978  Mission  Assurance  Conference. 

That  conference  set  the  tone  and  direction 
of  the  Space  Division  space  parts  program 
activities  of  the  past  2  years.  The 
remainder  of  this  presentation  will  report 
the  current  status  and  planned  direction 
of  the  Space  Division  space  parts  program. 

Parts  Panel  Highlights 

a.  Piece  part  reliability  critical 
to  space  system  mission  success 

b.  Program  part  reliability  problems 

(1)  Significant  part  failure/ 
yield  problem 

(2)  Program  peculiar  spec, 
requi rements 

(3)  Lack  of  available  standard 
space  quality  parts 

c.  More  comprehensive  parts  manage¬ 
ment  required 

d.  Newly  published  SAMSO  parts 
policies 

e.  DESC  committed  to  standardization 
of  Class  S  specs/parts 

f.  Well  managed  coordinated  procure¬ 
ment  offers  advantages 

g.  Joint  SAMSO/NASA/DESC  Class  S 
approach  being  implemented 

h.  Class  S  program  needs  accelera¬ 
tion  to  support  programs 

First,  however,  so  that  we  are  all  working 
from  a  common  baseline,  I  will  summarize 
the  objectives  of  the  SD  Space  Parts  Pro¬ 
gram  activities.  These  objectives  are 
stated  in  their  order  of  importance  to 
SD  missions,  although  most  of  the  indivi¬ 
dual  efforts  and  products  satisfy,  at 
least  partially,  more  than  one  of  the 
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listed  objectives. 


typical  of  space  programs. 


a.  Improve  parts  reliability 

(1)  All  part  tyoes 

(2)  Reduce/prevent  failures/ 

causes 

(3)  Standard  space  quality 
(Class  S)  documents  and  parts 

(4)  GSI/CSI 

b.  Improve  availability  of  space 
quality  parts 

(1)  Standardization  on  parts 
and  requirements 

(2)  Line  certifications 

(3)  MIL  QPL  (Class  S) 

(4)  Coordinated  procurement 

c.  Improve  parts  management  tech¬ 
niques  and  implementation 

d.  Reduced  program  costs 

(1)  Reduced  failures 

(2)  Spec  writing/negotiations 

(3)  Qual  tests 

(4)  Process  problems/yields 

(5)  Retest/ rework 

(6)  Schedule  slips 

The  Space  Division  approach  to  Improving 
reliability  and  availability  of  parts  for 
space  system  application  includes  the 
following  elements: 

a.  Extensive  management  by  each 
program  and  prime  contractor  of  all  parts 
used  in  the  system.  This  includes 
implementation  of  standard  part  management 
program  requirements  and  techniques. 

b.  Development  and  Implementation  of 
standard  "space  quality"  technical  (speci¬ 
fication  and  application)  requirements 
for  all  standard  and  non-standard  parts. 
This  includes  standard  test  methods  and 
identification  and  prevention  of  all  part 
failure  causes. 

c.  Development  of  standard  Class  S 
(space  quality)  military  parts,  specifica¬ 
tions,  certified  lines,  and  qualified 
products  lists. 

d.  Development  and  implementation  of 
requirements  and  methods  for  part  procure¬ 
ment  that  are  applicable  to  the  small 
quantity,  space  quality  parts  buys 


e.  Maintaining  the  Space  Parts 
Working  Group  as  an  SD/NASA/Government 
Agency/Industry  "Space  Part  Users" 
committee  to  formulate,  coordinate.  Inter¬ 
change,  and  assure  Integration  of  space 
parts  activities  and  products  for  Improving 
the  reliability,  availability,  and  cost 
effectiveness  of  space  parts. 

The  joint  SD/NASA/DESC/RADC  approach  to 
developing  Class  S  Military  Standard  parts 
and  sources  Is  summarized  below.  This 
approach  was  presented  and  endorsed  at  the 
1978  Mission  Assurance  Conference  and  has 
been  subsequently  applauded  by  the  Space 
Division  Advisory  Group  on  Electronic 
Reliability. 

a.  Identify/coordinate  candidate 


b.  Develop  joint  Class  "S"  (space 
quality)  requirements 

c.  Incorporate  Class  "S"  require¬ 
ments  Into  MIL  Specs 

d.  NASA/SD/DESC  certifications 

e.  DESC/NASA/SD  Class MlU" 
qualifications 

fl  SD/nASA  preferred  parts  lists 
g.  SD/NASA/DESC  baseline  control 

SD  Space  Parts  Program  Status 

The  current  SD  Parts  Policies  are  those 
policies  developed  just  prior  to  the  1978 
Mission  Assurance  Conference: 

a.  Select/procure  Class  "S"  parts 
if  available 

b.  Prime  contractor  conduct  coordina¬ 
ted  procurement 

c.  Monitor  manufacturer  processes 

d.  Assure  all  program  parts  meet 
program  reliability  goals 

When  MIL-Standards  1546  and  1547  are 
published,  we  will  expand  these  parts 
policies  to  include  implementation  of 
parts  program  management  and  standard 
technical  requirements  per  these  two 
documents. 

MIL-STD-1547  (Standard  Technical  Require¬ 
ments  for  Contractor  Specified  Parts)  is 
very  near  to  publication.  These  require¬ 
ments  were  developed  by  SD/AQT,  Aerospace 
Corporation  component  experts,  and  the 
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Space  Parts  Working  Group.  They  include 
additions  or  modifications  to  preferred 
MIL  Spec  requirements  and  are  based  on 
lessons  learned  and  failure  preventions 
from  previous/current  space  programs. 

These  standard  requirements  will  soon  be 
implemented  by  all  new  Space  Division 
programs. 

Summarized  below  is  the  progress  made  in 
the  availability  of  Class  S  microcircuits. 
Other  certifications  and  qualifications 
are  in  progress.  Filling  out  the  Class  S 
QPL  has  been  slow  due  to  uncertainty  of 
the  market,  infrequent  new  starts  (pro¬ 
grams),  and  the  shift  from  SSI/MSI  to 
LSIC  and  hybrids. 

a.  MIL-M- 38510  (Class  S  requirements) 
published 

b.  Class  S  MIL-STD-883  test  methods 
published 

c.  MIL- STD-976  (cert  requirements) 
published 

d.  7  certifications  complete 

e.  3  certifications  in  progress 

f.  Hardness  assurance  test  methods 
published 

g.  108  part  numbers  on  Part  II  QPL 

h.  Class  S  qual  effort  needed 

The  status  of  JAN  S  semiconductors  follows. 
Again,  qualification  is  the  current  hurdle 
and  we  plan  to  try  to  accelerate  the  JAN 
S  certifications  and  qualifications  in 
the  next  few  months. 

a.  MIL-S-19500F  (JAN  S  requirements) 
published 

b.  JAN  S  MIL-STD-750  test  methods 
published 

c.  Certification  requirements 
published 

d.  Motorola  small  signal  diode 
line  certified 

e.  JAN  S  qual  needs  acceleration 

The  following  summarizes  the  status  of  the 
highest  priority  Class  S  passive  spec 
projects.  These  efforts  will  receive 
increased  manpower  and  attention  for  the 
remainder  of  this  fiscal  year. 

a.  Ceramic  capacitors  -  MIL-C-87151 
(NASA)  -  CKS 

(1)  Final  draft  in  coordination 


b.  EMI  filters 

(1)  SD/NASA  spec  being  merged 
with  Navy/DESC  spec 

c.  Relays  (0-25  amps) 

(1)  Draft  Class  S  spec  in  work 
by  SPWG  subcommittee 

d.  Thermal  switches  -  MIL-S-24236 

(1)  Draft  Class  S  appendix  to 
MIL  Spec  in  work 

e.  MICA  Capacitors  (MIL-C-39001) 

(1)  Draft  changes  to  MIL  Spec 

in  work 

f.  Solid  tantalum  capacitors  (MIL- 
C-39003) 

(1)  NA.SA/Crane  testing  in  progress 

g.  Metalized  film  capacitors  -  MIL- 
C-83421 

(1)  NASA/SD  preparing  changes  to 

MIL  Spec 

The  SD  Preferred  Parts  list  have  been 
revised,  the  Class  S  Part  II  QPL  micro- 
circuits  added,  and  recently  published 
as  SD-STD  73-4B.  When  MIL-STD-1546  is 
published,  the  SD  PPL  will  be  appendices 
C  and  D  thereof. 

When  published,  MIL-STD-1546  (Parts  Pro¬ 
gram  Management  Requirements  for  Space 
and  Launch  Vehicles),  along  with  MIL-STD- 
1547,  will  replace  SAMSO-STD  73-2C. 

MIL-STD-1546  expands  and  clarifies  the 
Parts  Program  Management  Requirements  of 
SAMSO-STD  73-2C  and  includes  implementa¬ 
tion  of  SD  Parts  Policies,  Coordinated 
Procurement  requirements,  part  applica¬ 
tion  requirements,  and  the  SD  Preferred 
Parts  List.  Since  MIL-STD-1546  is  a 
Management  Control  System,  it  must  be  sent 
to  DOD/MIAG  for  review.  Publication  is 
expected  prior  to  1  August  1980. 
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Coordinated  Procurement  (by  the  Program 
Prime  Contractor)  of  electronic  parts, 
particularly  microcircuits  and  semi¬ 
conductors,  has  been  a  Space  Division 
policy  for  the  past  two  years.  Imple¬ 
menting  requirements  are  about  to  be 
published  in  Appendix  A  to  MIL-STD-1546. 

SD  programs  (DSCS  III,  lUS,  AMARV,  and 
ABRV)  have  Implemented  tailored  versions 
of  coordinated  procurement  and  have 
achieved  cost  and  schedule  benefits. 
Coordinated  parts  procurement  offers 
advantages  for  cost,  schedule,  and  relia¬ 
bility  if  adequately  planned  and  well 
managed.  Papers  and  discussions  at  both 
the  previous  Mission  Assurance  Conference 
and  this  workshop  explore  recent  experi¬ 
ences  in  implementing  various  forms  of 
coordinated  procurement,  advantages  and 
benefits  achieved,  and  Implementation 
difficulties.  Better  definition  of  and 
alternatives  to  Coordinated  Procurement 
is  one  of  the  three  principle  discussion 
topics  of  this  workshop. 

Government  Source  Inspection  is  one  of  the 
features  of  the  MIL-Spec  standard  part 
system.  The  Class  S  requirements  Include 
specific  GSI  inspections  as  a  condition 
of  JAN  marking  the  parts  and  releasing 
the  lot  for  delivery.  These  requirements 
are  contained  in  MIL-M-38510D,  MIL-S- 
19500F,  and  other  "Class  S"  MIL  specs. 

The  detailed  delegation  of  the  MIL-M- 
38510D  GSI  inspections  is  currently  in 
review  by  DLA  and  is  expected  to  be 
published/ implemented  in  the  near  future. 

Contractor  Source  Inspection  of  non-Class 
S  parts  is  required  by  MIL-STD-1546.  A 
better  definition  of  the  specific  CSI 
requirements,  by  part  type,  is  currently 
planned  for  development  by  SD/Aerospace 
and  the  SPWG  this  summer. 

At  the  1978  Mission  Assurance  Conference 
Microelectronics  Workshop,  Large  Scale 
Integrated  Circuits  and  Custom  Hybrid 
Microcircuits  were  identified  as  "new" 
part  types  finding  increasing  application 
in  space  systems.  Because  of  their 
increased  complexity,  lack  of  maturity, 
incomplete  testability,  divergence  between 
commercial  LSIC  and  space  LSIC  require¬ 
ments,  and  custom  circuit  development 
risks,*  large  scale  Integrated  circuits 
represent  a  significant  new  reliability 
risk  to  space  systems  and  therefore  require 
accelerated  effort  to  develop  Class  S 


requirements,  new  test  methods,  design 
rules,  more  complete  characterization, 
information  exchange,  procurement  tech¬ 
niques,  standardization,  and  technology 
developments. 

Since  the  1978  Microelectronics  Workshop, 
Space  Division  and  Aerospace  Corporation 
have  Increased  their  activities  to  define 
and  develop  space  quality  LSIC. 

a.  The  LSIC  committee  of  the  Space 
Parts  Working  Group  was  formed  and  is 
proceeding  in  the  development  of  LSIC 
requirements  definition,  alternate  screen 
tests,  improved  testability,  and  other 
LSIC  improvements. 

b.  The  LSIC  section  was  added  to  the 
SD  Preferred  Parts  List. 

c.  SD/Aerospace  have  participated 
with  RADC,  JPL,  and  others  in  the  develop¬ 
ment  of  MIL-STD  LSIC  slash  sheets. 

d.  Three  part  manufacturer  lines 
that  produce  LSIC  memory  devices  have 
been  certified  to  Class  S  requirements. 

e.  We  have  participated  with  RADC  in 
improving  characterization  of  selected 
MIL-STD  LSIC. 

Custom  Hybrid  Microcircuits  were  also  a 
principle  discussion  topic  at  the  1978 
Microelectronics  Workshop.  Like  LSIC, 
these  devices  offer  weight  and  performance 
advantages  but  can  also  introduce  cost, 
schedule,  and  reliability  risks.  The 
1978  Conference  scoped  the  Hybrid  situa¬ 
tion  and  recommended  we  get  on  with 
development  of  Class  S  requirements.  The 
SPWG  Committee  on  Hybrid  Requirements  was 
already  in  the  first  stages  of  the  task 
at  that  time. 

Today,  we  are  nearing  completion  of  Class 
S  requirements  for  Custom  Hybrid  Micro- 
circuits.  MIL-STD-1581 ,  which  contains 
these  requirements,  is  currently  in  review 
by  the  Space  Parts  Working  Group.  When 
completed,  this  document  will  be  Integrated, 
if  possible,  with  NASA  and  RADC  (Singer) 
prepared  documents  addressing  Custom  Hybrid 
Microcircuits.  Also,  since  the  DSCS  III 
program  used  many  custom  hybrids,  the 
experiences  and  lessons  learned  of  that 
program  will  be  summarized  for  use  in 
refining  our  requirements  and  approaches. 
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One  of  the  principle  topics  of  this  work¬ 
shop  will  be  approaches  and  recotmienda- 
tions  for  adequate  management  of  LSIC  and 
custom  hybrid  microcircu'lts.  Valid 
recommendations,  from  whatever  source, 
will  be  coordinated  and  included  in  MIL- 
STD-1546. 

There  is  much  more  work  to  be  done  to 
define  and  develop  standard  approaches 
and  requirements  for  LSIC  and  Hybrids. 

SD  and  Aerospace  intend  to  devote 
increased  resources  and  emphasis  to  these 
tasks.  We  hope  that  this  workshop  and 
subsequent  SPWG  activities  will  better 
define  the  tasks,  practical  solutions, 
and  specific  requirements  associated  with 
acquiring  space  quality  LSIC  and  custom 
Hybrid  microcircuits. 

Some  of  the  remaining  challenges  to  achieve 
space  quality  in  all  part  types  and  improve 
space  mission  reliability  are  listed  below. 
This  workshop  should  help  define  and 
prioritize  the  recommended  solutions  to 
these  and  other  challenges  surfaced  here. 
SD,  Aerospace,  and  the  SPWG  are  dedicated 
to  refining  and  implementing  these  and 
otner  recommended  solutions  for  the  benefit 
of  all  space  missions. 

a.  Longer  required  spacecraft  life¬ 
times 

b.  Increased  system  part  count  and 
part  density 

c.  Rapid  completion  of  Class  S 
documents 

d.  Building  Class  S  QPL(s) 

e.  Defining  the  "Class  S"  market 

f.  Defining/implementing  GSI/CSI 
and  other  "policing"  methods 

g.  Developing  implementable  "back¬ 
off"  approaches 

h.  Validating  reliability  of  new 
technologies 

i.  Developing  "Class  S"  technical 
and  management  requirements  for  LSIC  and 
Hybrids 

j.  Consistent  implementation  of 
"Class  S"  requirements 

k.  Improving  cost/delivery  times 
for  "Class  S"  parts 

l.  Resolving  SD/NASA  parts  approach 
divergences 

m.  Improving  flow  down  of  parts 
requirements  to  subcontractors 
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NASA  PARTS  MANAGEMENT 


Robert  C.  Karpen 
Manager  Parts  &  Materials 
Office  of  the  Chief  Engineer 
NASA  Headquarters 

NASA  parts  management  has  evol¬ 
ved  and  grown  over  the  past 
twenty  years  and  there  is  more 
growth  and  change  coming.  I 
will  not  dwell  on  our  past  but 
the  main  change  has  been  from  a 
solely  technical  concern  to  an 
expanded  involvement  in  parts 
man agement. 

TO  prepare  you  for  our  walk 
through  the  NASA  parts  manage¬ 
ment  system,  let's  first  do  a 
little  roadmapping  as  to  the 
players  and  documents  involved. 
(First  Chart) 

There  are  five  management 
functions  that  I  will  describe 
witn  respect  to  this  abbrevia¬ 
ted  organization  chart; 
o  policy  coordination 
o  policy  Surveys 
o  NASA  Parts  Steering 
Committee 
o  GIDEP/Alerts 
o  standard  Parts  Program 

Policy  concepts  originate  at 
various  sources  within  and  out¬ 
side  of  NASA.  Usually  they  are 
submitted  to  the  appropriate 
cognizant  office  which  will 
draft  and  coordinate  the  policy. 
Most  of  the  parts  management 
policies  are  derived  from  R&OA 
policies  or  recommendations  of 
the  NASA  Parts  Steering  Com¬ 
mittee.  The  extent  of  coordina¬ 
tion  depends  on  the  subject  and 

number  of  officials  assigned 
responsibility.  There  are  two 
levels  of  policy;  one  that  is 
NASA-wide  and  one  at  the  Center 

♦Government  industry  Data  Exchange 


level,  even  at  Headquarters. 
Consequently,  each  Center  can 
implement  parts  policy  differ¬ 
ently  as  long  as  the  Center- 
level  policy  is  not  in  conflict 
with  the  NASA-wide  policy. 

Periodic  surveys  of  the  Field 
installations  are  conducted  by 
Headquarters  to  assess  Center 
conformance  to  NASA  and  Center 
level  policy.  The  Chief  Engin¬ 
eer's  Office  surveys  Program 
Assurance  activities  at  Field 
Centers  every  two  years.  Special 
surveys  are  conducted  in  the 
field  and  at  contractor  plants  as 
needed. 

originally,  the  NASA  Parts 
Steering  Committee  was  formed  to 
exchange  parts  information  and 
resolve  technical  problems. 

Parts  management  problems  and 
techniques  are  appearing  more 
frequently  on  the  agenda.  Each 
Field  Center  has  a  representative 
as  do  most  Program  offices  at 
Headquarters.  Parts  policy  con¬ 
cepts  are  formed  and  debated  by 
this  Committee.  The  policy 
development  typically  is  tasked 
to  its  chairman  -  myself  -  for 
coordination  and  implementation. 
Three  of  the  policies,  that  have 
been  implemented,  concern  data 
exchange,  problem  reporting  and 
standardization.  They  define 
NASA  participation  in  GIDEP*, 
the  Alert  Problem  Reporting 
System,  and  the  NASA  Standard 
Parts  Program. 

(Chart  2)  The  standard  Parts 
Program  has  been  established  to 
promote  the  use  of  standard  EEE 
parts.  A  Lead  Center  office 
was  established  at  Marshall 
Space  Flight  Center  (MSFC)  in 
the  R&QA  Office  and  operates 
the  program  with  technical 

Program 


support  both  from  within  MSFC, 
from  other  Centers,  Naval 
Weapons  Support  Center  and 
tc;st  and  engineering  support 
contractors.  The  office  is 
responsible  for  implementing 
standardization  projects  and 
coordination  activities  with 
other  NASA  Centers,  DOD  and 
industry . 

Next,  a  quick  look  at  the 
principal  policy  documents 
influencing  parts  management 
within  NASA.  (Chart  3)  One 
of  the  basic  controls  is  the 
Reliability  and  Quality 
Assurance  policy.  Through 
it  the  requirements  documents 
in  the  NHB  5300.4  series  have 
been  developed  (Chart  4) ,  It 
is  required  by  our  procurement 
regulations  that  these  require¬ 
ments,  be  reviewed  and  included 
as  contract  requirements,  as 
appropriate.  special  policy 
and  Project  requirements  also 
been  developed  for  special 
situations.  For  example, 

NHB  5300.4(10-2)  for  the 
Shuttle  Program,  NMI  8010.1 
Classification  of  NASA  Space 
Transportation  System  pay- 
loads,  and  special  parts 
lists  and  control  documents 
for  projects. 

Referring  back  to  a  previous 
chart,  (Chart  3)  you'll  note 
that  there  is  no  direct  tie 
from  our  standardization 
policy  through  our  procure¬ 
ment  regulations  to  our 
contracts.  New  parts  policy 
cirafts  are  in  process  to 
implement  this  and  other, 
more  recent,  parts  controls 
which  are  not  contained  in 
the  NHB  5300.4  scries. 

BASIC  PRINCIPLES 

What  arc  some  of  the  basic 


principles  we  use  in  the 
NASA  parts  program?  One  is 
"Don't  Reinvent  the  wheel!". 

We  can't  afford  to.  There 
are  many  examples  where  we 
share  information.  There 
are  problem  reports,  experi¬ 
ence  bulletins,  test  reports, 
calibration  procedures,  speci¬ 
fications,  standards,  test 
methods,  guidelines  and  on, 
and  on.  Cost  sharing  is  also 
important  in  the  services 
area.  we  share  services, 
such  as  DCAS  and  AFPRO  at 
our  contractors  sites,  DESC 
for  spec-if ications  and 
standards,  and  the  Navy 
publications  capabilities. 

There  can  be  problems  with 
other  people's  documents  and 
services.  They  can't  be 
blindly  applied  to  any  new 
situation.  There  are  many 
times  that  they  will  have  to 
be  modified  to  make  them  so 
they  are  more  palatable  when 
you're  bucking  a  "not-invented- 
here"  or  "not-in-our-format" 
syndromes. 

The  second  principle  is  also 
an  attempt  to  hold  down  the 
costs.  we  advocate  the 
selection  of  parts  on  the 
basis  of  minimum  overall 
project  cost  to  NASA,  not 
just  the  front-end  cost  as 
listed  in  the  parts  vendors 
catalog.  It's  not  easy. 

High  reliability  parts  are 
expensive.  Use  of  estab¬ 
lished  aerospace  contractors 
in-house  spec  systems  has  its 
advantages.  The  use  of 
commercial  parts  and  rescreen- 
ing  appears  cheap.  What  we 
need  is  more  data,  like  a 
Goddard  Space  Flight  Center 
evaluation  two  years  ago  of 
Class  A  versus  Class  B  versus 
commercial  parts  in  terms  of 


total  cost  but  also  with  prob¬ 
able  success  estimates.  It 
can  show  project  managers, 
and  I  think  parts  people,  the 
relative  cost  at  each  point 
and  net  impact  of  their 
selections.  This  need  for 
assessment  is  not  only  true 
in  the  part  quality/reliability 
areas  but  in  parts  procurement 
techniques  and  project  schedules. 
Programs  are  starting  to  release  ' 
funds  earlier  to  permit  purchase 
of  long  lead  items.  The  NASA 
project  manager  is  still  the 
prime  decision-maker  on  hard¬ 
ware  and  is  given  a  fair  amount 
of  leeway  in  his  pursuit.  The 
expanded  documentation  and  data 
is  geared  toward  helping  him 
make  the  assurance  decisions. 

How  often  did  you  hear  that 
designing  with  military  parts  is 
designing  with  obsolete  parts? 
We're  caught  in  an  unusual, 
and  I  think  fantastic, 
electronics  evolution.  I 
don't  want  to  stymie  creativ¬ 
ity  and  innovation.  But,  let's 
use  the  edge-of-technology 
devices  where  they  are  not 
critical.  The  designers  are 
trying  to  hurry  the  new  devices 
into  the  hardware.  we're 
trying  to  hurry  the  cycle, 
too.  Through  the  standardiza¬ 
tion  Program  we  are  helping 
mature  new  technologies  by 
characterizing  and  documenting 
early  in  their  development. 

Our  independent  product  evalu¬ 
ation  has  resolved  design 
errors  sooner  than  if  we  had 
waited  for  the  design  to 
stabilize  on  the  commercial 
market.  in  spite  of  their 
complexity,  I  believe  the 
microprocessors  are  getting 
specified  faster.  in  addition 
to  the  products,  we  are  start¬ 
ing  to  scrutinize  test  methods 
better  and  we'll  be  analyzing 
parts  management  techniques  so 


that  there  is  a  shorter  trial 
and  error  period. 

The  last  principle  is  the  old 
"Don't  bite  off  more  than  you 
can  chew".  we 'ye  been  evolv¬ 
ing  into  a  parts  program  over 
several  years.  It  started 
by  concentrating  on  the  micro- 
circuit  devices,  later  expanded 
to  most  EEE  parts,  and  is 
expected  to  spawn  an  equiva¬ 
lent  materials  program  soon. 

But  as  we  have  evolved,  we 
have  been  maintaining  the  many 
facets  that  we  have  developed. 
So  many  good  ideas  have  fallen 
by  the  wayside  because  it's 
more  fun  to  create  concept  than 
to  maintain  it.  Maintenance 
is  hard  to  support  when  priori¬ 
ties  tend  to  shift  alot.  We 
are  trying  to  handle  those 
shifts  by  also  remaining  flexi¬ 
ble. 

WHAT'S  MISSING 

Here  are  some  of  the  areas 
we'll  be  working  on  and  a 
couple  that  are  still  problems. 
We  are  developing  stronger 
management  of  the  parts  program 
It  has  grown  to  the  point  where 
better  visibility  and  controls 
are  needed  to  keep  it  efficient 
We  are  building  a  parts  usage 
data  base  to  forecast  trends 
and  establish  priorities. 

Hopefully,  we  can  centralize 
parts  problem  reporting  to 
effect  fundamental  corrective 
actions  to  specs,  standards 
and  procedures  when  needed. 

I  think  we  need  better  visibil¬ 
ity  of  NASA  procurement  of 
Class  S  level  parts  and  the 
effectiveness  of  the  new  pay- 
loads  policy. 

We  will  start  looking  at  ways 
to  more  accurately  define 


what  we  really  need.  Better 
evaluation  of  new  test  methods, 
assembly  methods,  and  manage¬ 
ment  techniques  to  determine 
their  risks  and  effectiveness. 
What  are  good  techniques  to 
get  realistic  user  requirements? 
A  related  area  is  establishing 
priorities  among  the  multitude 
of  tests,  evaluations,  docu¬ 
ments,  etc.  priorities  used 
by  different  organizations 
vary  for  obvious  reasons. 

Some  of  our  conflicts  in  the 
parts  program  are: 

o  Grade  1  versus  Grade  2 
development 

o  Help  small  quantity  or 
large  quantity  user 
o  Build  in  reliability 
versus  screening  out 
defects 

As  my  last  comment,  let  me 
toss  out  a  thought  that  might 
get  us  a  stock  of  qualified 
parts,  where  are  the  spare 
parts  from  many  of  our  cost- 
plus  programs.  Can  they  be 
made  available?  Could  we 
have  cin  initial  phase  that 
would  list  spares  and  their 
source,  a  second  phase  with 
a  central  repository  and  that 
could  be  followed  by  a  replen¬ 
ishment  program;  that  sounds 
like  stocking  by  another  name. 
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MILITAK  Y/A1';K0SPA(;K  market 
John  D.  Shea 

INTRODUCTION 

Over  the  last  eighteen  months,  the 
military  needs  for  advanced  high- 
technology  solid-state  integrated 
circuit  devices  has  become  focused 
before  the  public  eye. 

Because  of  the  continued  debate  with 
regard  to  the  ratification  of  the  SALT 
II  Treaty  by  the  Congress  of  the  United 
States,  a  definite  question  remains  as 
to  whether  the  continued  discussions 
between  the  United  States  and  the  Soviet 
Union  will  act  as  a  diplomatic  break¬ 
water  to  limit  the  spread  of  high-tech¬ 
nology  weapon  systems. 

The  Department  of  Defense  planners 
are  realizing  that  LSI  solid-state  in¬ 
tegrated-circuit  devices  will  continue 
to  becom.e  a  key  element  in  force 
multiplication.  The  need  for  real-time 
numerical  generation,  computation, 
signal  processing,  and  sensing  is  tax¬ 
ing  existing  silicon  technology. 

The  emphasis  placed  by  the  USSR  on 
satellite  space  weaponry  systems  for 
reconnaissance  secured  communication, 
2nd  tatical  deployment  of  nuclear  war¬ 
heads  for  reentry  at  selected  targets  on 
earth,  has  created  a  need  for  closer 
cooperation  between  the  Department  of 
Defense  (DoD),  the  industrial  comm¬ 
unity,  &  the  academic  environment 


within  tlu-  United  State's.  (See  Taljle 
S-  I) 

With  the  realization  of  advances  in 
Soviet  technology,  as  experienced  by 
analysis  of  equipment  captured  by  the 
Israelis  during  the  1973  Y om  Kippur 
War,  it  became  evident  that  the  Soviet 
Union  had  reached  near  parity  with  the 
United  States  in  key  technological  areas 
of  infrared  detection  and  sensing,  as 
well  as  sophisticated  utilization  and 
application  of  high-technology  IC  de¬ 
vices. 

Key  areas  receiving  continued  attention 
from  the  Department  of  Defense  and 
supported  by  selected  semiconductor 
manufacturers  and  prime  military/ 
aerospace  systems  manufacturers  are: 

A)  VHSIC  (very  high-speed  integration 
IC  devices) 

B)  Radiation-hardened  technology  for 
IC  devices 

C)  CCD  {cha rge- coupled)  IC  imaging 
devices  used  for  infrared  detection, 
secure  communication  satellites  and 
spy-in-the-sky  satellites. 

The  Soviet  Union  has  supplied  to  re¬ 
presentatives  of  Control  Data  Corp¬ 
oration  examples  of  Soviet  semiconduct¬ 
or  devices  that  were  direct  copies  of  U. 
S.  products.  It  becomes  apparent  that 
even  if  these  devii.es  were  laboratory 
experimental  units,  the  Soviets  tiave  the 
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■  ilnhly,  il  Ihi-y  so  drsii'o,  to  a]jply  ri'- 
soiiis  fs,  1 1 1.1  n [)o\\  I ■  r ,  anti  ftintliny  to 
.onipflo  .1  u  u  I'f  s  s  I  \  o  1  y  in  l  ho  military/ 

.1  o  rosp.u  o  cnv  i  ronnu'iit  w  ith  solitl- 
sl.ilo  t  ft  lin  oloy  y  .  riu-  .So\’iot  a  t  Ka  n  co  s 
in  .MIKV'  warhf.ul  satollito  miasilt*- 
(lisponsinp  systems  already  in  plaee  in 
orbit  .1  rounti  the  earth,  as  well  as  the 
So\iels'  i  onlimied  s ophi sli ea t i on  in  re- 
»  onn.i  1  s  s<uu  e  satellites,  make  it  app- 
, I  rent  that  the  I  nitetl  States  has  no  re- 
t  ourse  hot  to  alloeate  the-  necessary 
lunditiL;  and  resourt  i's  to  assure  main- 
ten, inti-  of  parity  anti  its  technological 
snpe  rio  rit  y. 

Several  key  programs  are  underway 
within  the  Department  of  Defense  )st 
will  be  reviewed  this  section.  The 
V'llSIC  program  is  the  cornerstone  of 
the  altility  of  the  United  States  to  use 
solid-state  tlevices  as  a  force  multi¬ 
plier,  allowing  battlefield  commanders 
to  make  real-time  tactical  decisions 
affecting  spaceborne,  airborne,  ship- 
borne  and  landborne  forces.  In  the 
area  of  radiation-hardened  devices, 
several  companies  such  as  Lockheed, 
Rockwell,  Hughes  Aircraft,  National 
■Semiconductor,  and  Harris  Semicon¬ 
ductor  have  been  engaged  in  productive 
research  to  harden  aerospace  weapon 
systems. 

The  manufacturing  technology  program 
supported  Ijy  the  Navy,  Army  k  Air 
f'orce  with  regard  to  the  "electronics 
wi'dge  "  scoping  will  take  advantage  of 
advant  ed  state  of  the  art  wafer-pro- 


l  essing  technology.  The  c  ompletely 
comput e  r- cont  rolled  automated  fac  tory 
IS  one  possible  result  of  this  program. 
Tills  facility  would  provide  multiple- 
technology  wafer  processing  under 
computer-controlled,  minimal  contam¬ 
ination  eiic  i  ronment  s  to  manufacture 
high-density  complex  integrated  cir¬ 
cuits  of  the  highest  level  of  field  re¬ 
liability  and  integrity. 

Never  before  in  the  history  of  the  United 
States  have  strategic  military  impli¬ 
cations  with  regard  to  national  security 
been  more  dependent  on  the  design, 
manufacturing,  and  application  of  high- 
technology  integrated  circuits  as  now. 
Table  5-2  reviews  the  I'.  S.  semicon¬ 
ductor  manufacturers'  support  of  DoD/ 
Aerospace  programs  fc-  technologies. 

THE  VHSIC  PROGRAM 

INTRODUCTION 

The  Dob)  has  initiated  a  major  new  pro¬ 
gram  in  high  speed,  high  throughput 
signal  and  data  processing  in  support 
of  the  requirements  for  military  sys¬ 
tems  in  the  mid-eighties  &  beyond.  The 
program  title  is  Very  High  Speed  Inte¬ 
grated  Circuits  (VHSIC)  which  emphas¬ 
izes  its  significant  relationship  to  the 
technology  of  ICs  as  well  as  the  need 
for  higher  speed  processing  capability. 

Successful  completion  of  the  VHSIC  pro¬ 
gram  will  enable  the  DoD  to;  provide 
critical  electronic  subsystems  required 


to  meet  military  shortcomings  ex¬ 
pected  in  the  mid-1980's  and  again  in 
the  early  1990' s,  retain  &  extend  the  US 
technological  leadership  in  advance 
military  electronics;  reduce  life  cycle 
costs  associated  with  military  elec¬ 
tronics  systems;  and  avoid  a  severe 
future  problem  in  utilizing  advanced 
ICs  in  military  systems  due  to  the  fact 
that  they  will  be  either  not  available, 
not  affordable  or  both,  or  not  meet 
military  specifications. 

VHSIC  -  Acronym  for  Very  Highspeed 

integrated  Circuits.  This  acronym  was 

developed  to  describe  the  program  to 

develop  very  large  scale  integrated 

circuits  applicable  to  the  DoD  needs, 

particularly  very  high  speed  signal 

processing.  The  speed-density  product 

1  3 

figure-of-merit  goal  is  10  gate  -Hz/ 
cm  .  The  acronym  is  also  used  to  de¬ 
scribe  the  integrated  circuits  that  will 
be  developed  under  the  program. 

Background 

The  initiation  of  the  VHSIC  program 
represents  a  major  reversal  of  past 
DoD  policy  to  depend  on  industry  to  de¬ 
velop  needed  ICs.  Present  judgment  is 
that  industry  is  not  presently  oriented 
towards  meeting  the  DoD's  current  & 
future  needs.  The  industry  trend  is 
directed  towards  very  high  volume 
commercial  applications,  thus  per¬ 
mitting  high  initial  design  costs  to  be 
written  off  over  many  production  units. 
The  DoD  market  now  constitutes  about 


seven  per  cent  of  the  US  total  IC  mar¬ 
ket.  .  .  .  with  many  low -volume  appli¬ 
cations  requiring  the  ultimate  in  per¬ 
formance.  The  industry  trend  is  toward 
general-purpose  ICs  to  be  used  in  com¬ 
mercial  data  processing  and  consumer 
applications.  It  is  not  sufficiently 
oriented  toward  qualifying  ICs  for  mili¬ 
tary  specifications  or  toward  develop¬ 
ing  ICs  with  clock  speeds  sufficiently 
high  to  perform  current  and  projected 
real-time,  high-speed  signal  process¬ 
ing  functions.  Furthermore,  in  con¬ 
trast  to  many  commercial  applications, 
on-chip  self-test,  self-check,  and 
fault-tolerant  techniques  are  import¬ 
ant  needs  to  provide  acceptable  logis¬ 
tics  costs  to  the  military. 

The  VHSIC  program  differs  significant¬ 
ly  from  the  bread  commercial  VLSI 
thrust  in  integrated  circuits  in  the 
following  major  respects: 

1.  There  will  be  major  emphasis  on  IC 
development  for  broad  classes  of  mili¬ 
tary  systems--to  develop  technology 
and  deliver  products  for  which  there  is 
no  comparable  commercial  or  indust¬ 
rial  need; 

2.  There  will  be  major  emphasis  on 
achieving  new  architectural  concepts 
which  will  minimize  the  need  for  de¬ 
sign  customization  and  hence  reduce 
costs  in  design,  supply  &  logistics; 

3.  There  will  be  major  emphasis  on 
increasing  real-time  system  throughput. 
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higher  chip  complexity  and  higher  clock 
rates  to  achieve  a  capability  for  real¬ 
time  signal  processing. 

4.  There  will  be  major  emphasis  on 
military  environmental  requirements 
such  as  performance  over  a  high  temp¬ 
erature  range,  radiation  exposure  and 
reliability.  There  will  be  exploitation 
of  the  higher  chip  complexity  available 
to  achieve  on-chip  and  fault  tolerance 
with  significantly  improved  reliability 
and  mean-time-to- repair.  Emphasis  is 
placed  on  high  speed  signal  processing 
applications  having  considerable  com- 
patability  with  commercial  data  pro¬ 
cessing.  All  of  the  VHSIC  efforts  will 
take  advantage  of  the  leverage  provided 
by  the  larger  commercial  VLSI  activity 
but  will  concentrate  on  those  technology 
tasks  essential  to  achieving  military 
goals. 

Program  Description 

To  meet  the  above  DoD  objectives,  the 
program  is  divided  into  four  parts; 
Phase  O,  I,  II,  and  III,  each  with  dis¬ 
tinct  and  important  goals.  The  end  goal 
of  the  entire  program  is  to  reach  a  cap¬ 
ability  for  advanced  systems  perform¬ 
ance  based  on  availability  of  ICs  with 
submicrometer  feature  sizes.  Phase  O, 
I  &  II  will  be  carried  out  consecutively, 
while  Phase  III  will  be  carried  out  in 
parallel  with  Phases  O,  I,  and  II. 

Phase  O-  Refers  to  the  program  part 
called  "Program  Definition"  to  provide 


a  plan  and  approach  for  Phase  I  and  II. 
(See  Table  3) 

Phase  I  -  Refers  to  the  program  part 
Oriented:  (1)  to  the  construction  of  elec¬ 
tronic  brassboard  subsystems  by  about 
1983  using  VHSICs  based  on  1. 25  micro¬ 
meter  minimum  feature  sizes;  and  (2) 
to  carry  out  initial  work  for  achievement 
of  VHSICs  based  on  submicrometer  fea¬ 
ture  sizes. 

Phase  II  -  Refers  to  the  program  part 
oriented  (1)  to  the  system  demonstra¬ 
tion  of  the  electronic  brassboard  sub¬ 
systems,  and  (2)  the  achievement  of 
VHSICs  with  submicron  (nominally  0.  5 
to  0.  8  geometries)  by  about  1985. 

Phase  HI  -  Technology  efforts  which  are 
generic  in  nature  and  which  are  suppor¬ 
tive  of  Phase  I  and  Phase  II. 

Impact  of  the  DoD  VHSIC  Program  On 
The  Integrated  Circuit  Industry 

Several  questions  have  been  raised  per¬ 
taining  to  the  impact  that  the  DoD's 
VHSIC  program  will  have  on  the  inte¬ 
grated  circuit  industry.  Some  of  the 
more  thought-provoking  are: 

Will  the  VHSIC  program  further 
aggravate  present  manpower  shortages 
in  the  IC  industry  ? 

Could  the  VHSIC  program  greatly 
help  a  few  companies  and  thereby  skew 
the  marketplace  ? 
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Will  a  commercial  market  develop 
for  advanced  VHSICs  ? 

The  DoD  stimulated  the  growth  of  the 
IC  industry  in  the  1960s  by  demonstrat¬ 
ing  the  feasibility  of  fabricating  ICs, 
and  by  providing  a  viable  market.  This 
was  followed  by  a  rapid  expansion  of 
technology  through  government  reports, 
transfer  of  personnel  among  companies, 
and  establishment  of  second  source  and 
licensing.  In  some  cases,  once  one 
company  announced  or  introduced  a  new 
process  or  product,  the  same  result 
was  repeated  in  other  companies.  The 
DoD  feels  the  same  process  will  be  re¬ 
peated  to  a  considerable  extent  with 
VHSIC.  The  DoD  hopes  to  prove  the 
feasibility  and  provide  demonstrations 
of  VHSIC.  Some  of  the  program's  goals 
themselves  will  help  to  insure  that  a 
DoD  market  will  rapidly  accrue.  The 
near-term  goal  of  the  1.  3  micron  design 
rule  could  coincide  with  commercial 
markets  in  1982.  This  could  also  stim¬ 
ulate  commercial  production.  ICs  that 
are  developed  under  the  VHSIC  program 
will  be  MIL-Spec  qualified,  allowing 
immediate  DoD  use. 

The  specific  VHSIC  team  members  & 
proposed  technology  offerings  have  nc>t 
been  officially  disclosed  as  of  this  date. 
Table  5-4  lists  the  teaming  member 
combinations  mentioned  in  the  press  as 
most  probable. 


interconnected,  addressed  and  pro¬ 
grammed  to  meet  design  objectives  of 
application  algorithms.  This  efini- 
tion  includes  interconnection  or  busing, 
instruction  set,  special  functions,  con¬ 
trol,  and  addressing  I/O  schemes,  all 
compatible  with  an  operating  system 
including  executive  and  language  pro¬ 
gramming. 

Lithography 

It  is  expected  that  Phase  la  chip  tech¬ 
nology  will  be  satisfied  with  presently- 
available  lithography  equipment.  De¬ 
velopment  of  advanced  lithography 
equipment  for  submicron  fabrication 
(such  as  e-beam  or  X-ray  equipment) 
can  be  i  icluded  as  part  of  Phase  lb  in 
a  vertically  integrated  effort,  or  can 
be  included  as  part  of  Phase  III  in  re¬ 
sponse  to  an  P.FP,  or  need  not  be  in¬ 
cluded  in  either.  This  will  be  an  option 
of  the  offeror.  If  bid  as  part  of  a  ver¬ 
tically  integrated  effort,  it  will  be 
clearly  separated  from  other  efforts. 
Simplified  program  goals  are  listed  as 
a  function  of  lithography  feature  sizes 
in  Table  5-5. 

JAN  38510  Class  B  Market 
In  the  integrated  circuit  industry,  the 
terms  "hi-rel"  and  "JAN"  are  often 
used  synonymously  and,  all  too  often, 
loosely.  The  MIL-M-38510  JAN  in¬ 
tegrated  circuit  program  was  estab¬ 
lished  in  1968  by  the  Rome  Air  Eievelop 
ment  Command  (RADC)  of  the  United 
States  Air  Force.  This  program  was 
lished  to  control  the  wafer  processing. 


VHSIC  Architecture 

A  structure  by  which  VHSIC  devices  are 
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assembly,  electrical  testing,  overall 
package  integrity  and  environmental 
testing  of  integrated  circuit  devices. 

The  controlling  Military  Standard  is  MIL 
MIL-STD  883,  established  in  1968.  On 
recent  military  programs  the  Depart¬ 
ment  of  Defense  has;  mandated  that 
MIL-M-38510  devices,  when  available, 
will  be  used  by  the  prime  and  sub¬ 
contractors  in  all  military  and  aero¬ 
space  contracts.  A  current  coordin¬ 
ation  between  the  National  Aeronautics 
and  Space  Administration  (NASA),  the 
Air  Force  Systems  Command  (SAMSO), 
and  RADC  has  consolidated  the  genera¬ 
tion  of  hi-rel  processing  specifications 
for  the  DoD  and  other  governmental 
agencies.  This  will  enable  both  NASA 
and  SAMSO  to  make  meaningful  quantity 
buys  that  will  be  profitable  for  the  semi¬ 
conductor  industry  to  manufacture.  The 
"S"  classification  is  one  attempt  with 
the  vendors  to  replace  the  unworkable 
Class  "A"  specification.  The  Class  B 
served  available  market  (SAM)  for 
digital  products  is  broken  out  by  \«ndor 
in  Table  5-6. 

PRELIMINARY  CLASS  S  MARKET 
NUMBERS 

Class  S  Jan  Market  (Space  Programs) 


The  Class  S  requirements  are  designed 
to  circumvent  known  problems.  In 
general,  each  requirement  matches 
up  to  a  problem  the  DoD  has  had  one  or 
more  times  in  the  past.  (These  are  not 
things  that  were  invented  without  any 
real  basis.  )  The  effectiveness  of  the 


Class  S  requirements  is  predicated  on 
the  implementation  of  all  of  them. 

There  aren't  any  magic  requirements 
which  can  be  implemented  to  the  ex¬ 
clusion  of  the  others.  The  DoD  is  de¬ 
pending  upon  the  implementation  of  the 
entire  specification  and  all  of  its  re¬ 
quirements  to  achieve  this  high  quality. 
Finally,  the  Class  S  requirements  were 
derived  from  specifications  which  the 
DoD  has  successfully  used  in  the  past 
on  prior  space  programs;  they  reflect 
screening  test  methods  which  have 
proven  effective  for  the  DoD.  The  de¬ 
sign  and  construction  rules  in  the  spec¬ 
ifications  are  there  to  preclude  known 
reliability  problems.  The  established 
control  techniques  in  the  specification 
have  been  used  by  the  DoD  in  the  past. 
All  of  these  requirements  have  received 
extensive  coordination  with  both  con¬ 
tractors  and  manufacturers.  The  total 
available  market  (TAM)  for  Class  S 
products  is  estimated  in  Table  5-7. 

SAMSO  has  summarized  the  Class  S  re¬ 
quirements  by  breaking  them  into  nine 
major  catagories: 

1.  Manufacturing  baseline. 

2.  Design  &  construction  re¬ 
quirements  . 

3.  Product  assurance  program. 

4.  Certification  of  manufactur- 
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5.  Class  S  qualification  re¬ 
quirements. 

6.  Wafer  lot  acceptance. 


Center  ,  DESC-EQ,  as  the  DoD  focal 
point  for  the  JAN  Microcircuit  program 
DESC-EQ  will  also  perform  all  admin¬ 
istrative  work  on  Class  S  devices. 


7.  Class  S  Screening  require¬ 
ments. 

8.  Quality  conformance  testing. 


9.  Government  Source  Inspec¬ 
tion  (GSI). 

The  evolvement  of  Class  S  devices 
under  MIL-M-38510D,  and  more  signif¬ 
icantly,  the  joining  of  DESC,  NASA 
and  SAMSO  to  effectively  administer 
and  invoke  the  Class  S  qualification  & 
certification  requirements,  confirms 
the  government's  belief  that  working  to¬ 
gether  on  this  issue  is  the  way  to  go. 
Wide  support  exists  at  the  command 
level  for  the  program.  In  the  past, 
DESC,  NASA  and  SAMSO  have  been  es¬ 
sentially  operating  independently  to 
quality  high  reliability  parts.  DESC 
has  administered  qualification  for  Class 
B  and  C  devices,  and  has  authorized 
testing  and  evaluated  test  reports  for 
the  old  Class  A  device  s.  NASA  has 
conducted  its  own  line-certification 
audits  for  the  old  Class  A  devices. 
SAMSO  is  administering  its  own  quality 
program  under  MIL-M-00385- lOB. 


The  method  by  which  DESC,  SAMSO  & 
NASA  will  work  together  in  the  Class  S 
qualification  program  will  be  reviewed 
by  the  Defense  Electronics  Supply 
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The  key  point  in  Class  S  qualification 
is  the  provision  for  a  joint  team  audit 
composed  of  individual  members  from 
each  government  activity  that  has  an 
interest.  This  will  result  in  several 
advantages; 

a.  Minimize  company's  time  spent 
during  audits. 

b.  Eliminate  audit  duplication  and 
overlap. 

c.  Reduce  overall  government  audit 
expense. 

d.  Improve  uniformity  of  audits. 

To  effect  this  last  advantage,  the  gov¬ 
ernment  plans  to  restrict  the  number  of 
participants  performing  the  audit  to  pro- 
vice  an  efficient  team.  Only  the  people 
essential  for  the  audit  will  be  in  atten¬ 
dance.  This  will  ensure  cost  effective¬ 
ness  and  provide  a  uniform  audit  in 
minimum  time. 

NASA/SAMSO  feel  that,  aside  from  the 
qualification  program,  other  goals  need 
to  be  accomplished.  With  the  incor¬ 
poration  of  Class  S  devices  in  MIL-M- 
38510D,  the  DoD  should  strive  to  estab¬ 
lish  Class  S  devices  as  preferred  for 
logistics  purposes.  This  would  mini¬ 
mize  the  number  of  varieties  of  devices 
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that  need  to  be  stocked  for  support 
purposes.  While  the  cost  of  Class  S 
devices  initially  would  be  somewhat 
higher,  an  increase  in  usage  may  make 
it  feasible  to  stock  only  the  Class  level 
devices  for  replacement  purposes.  In 
most  cases  the  net  result  would  be 
lower  life  cycle  costs. 

Selected  Military/Aerospace  Supplier 
Profiles 

The  following  supplier  profiles  outline 
specific  supplier  activities,  special 
interests  &  capabilities  &  U.  S.  sales 
in  the  military/aerospace  market. 

Texas  Instruments  (TI) 

TI  continues  to  support  the  Mil-M-3851 
38510  JAN  integrated  circuit  program, 
both  Class  B  and  Class  S.  TI  has  been 
a  leader  in  the  utilization  of  electron- 
beam  mask-projection  systems  that 
will  be  a  major  factor  in  providing  com¬ 
plex  device  geometries  in  the  sub¬ 
micron  (0.  5  micron)  feature  size  as  re¬ 
quired  for  the  VHSIC  program. 

National  Semiconductor 

National  Semiconductor  continues  to 
maintain  its  position  as  a  major  sup¬ 
plier  to  the  military/aerospace  market. 
National  will  continue  to  take  advan¬ 
tage  of  its  position  as  a  major  sup¬ 
plier  of  radiation-hardened  CMOS  and 
radiation-hardened  linear  integrated 
circuits.  National  has  had  a  major 


investment  to  support  the  VHSIC  pro¬ 
gram,  and  is  presently  one  of  the  few 
suppliers  presently  in  possession  of 
both  VLSI  and  rad-hard  capabilities. 

Fairchild  Semiconductor 

Fairchild  has  realigned  its  bipolar 
military  operations  and  has  established 
within  that  operation  that  bipolar  LSI 
military  hi-rel  strategic  business  unit, 
with  emphasis  to  be  placed  on  bipolar 
memories  and  RAMs.  Fairchild  plans 
to  participate  in  the  VHSIC  program. 

Signetics 

Signetics  continues  to  be  an  effective 
supplier  of  military/ae rospace  high- 
reliability  products  and  is  a  major 
participant  in  the  Class  S  space  parts 
38510  program.  Signetics  intends  to 
participate  in  the  VHSIC  program, 
teaming  with  Hughes  Aircraft,  utilizing 
Signetics'  injection  Schottky  logic  pro¬ 
cess.  Signetics  continues  to  secure 
qualifications  to  MIL-M-38510  for  its 
linear  analog  bipolar  devices  and  also 
continues  to  be  a  major  supplier  of  bi¬ 
polar  PROMs  to  the  military/aerospace 
marketplace. 

Motorola 

Motorola  has  established  a  bipolar 
high- reliability  operations  center  in  the 
Mesa,  Arizona  facility.  Motorola  con¬ 
tinues  to  process  MOS  &  CMOS  military 
products  at  the  Austin  facility. 
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One  of  the  key  problems  that  Motorola 
exhibits  is  one  of  duplicated  operations 
on  a  division-by-division  level,  as  it 
has  not  yet  established  a  centralized 
high- reliability  profit-  and-loss  center. 
Motorola  is  reportedly  teamed  with. 

TRW  and  Univac  on  the  VHSIC  program. 

Advanced  Micro  Devices 

AMD  has  elected  not  to  participate  in 
the  VHSIC  program,  primarily  because 
this  would  create  a  dilution  of  corporate 
resources,  especially  manpower.  AMD 
continues  to  support  the  JAN  (MIL-M- 
38510).  The  AMD  2901  bipolar  micro¬ 
processor  continues  to  be  a  standard 
for  military  bit-slice  computers. 

Harris 

Harris  has  been  long  known  for  its 
radiation-hardened  capabilities,  and 
Harris  is  involved  in  many  programs  of 
a  classified  nature  which  support  the 
National  Security  Agency  and  SAMSO  in 
the  area  of  recommaissance  and  secured 
communications  systems.  Harris  has 
made  a  major  commitment  to  support 
both  Class  B  and  Class  S  of  the  MIL-M- 
38510  program. 

Intersil 

Intersil  continues  to  support  its  Indepen¬ 
dent  Hi-rel  profit-and-loss  center. 

Intersil's  strengths  come  in  support  of 
the  precision  analog  linear  integrated 


circuit  market  segment,  as  well  as 
low-power  CMOS  as  applied  to  military 
and  aerospace  systems.  Intersil  is  re¬ 
portedly  teamed  with  GE  on  the  VHSIC 
program. 

RCA 

RCA  has  continued  to  support  the  MIL- 
M-38510  Class  B  and  Class  S.  RCA  is 
known  for  providing  good  engineering 
support  to  its  customers  and  to  various 
DoD  agencies.  RCA's  microprocessor, 
the  CDP-1802,  has  been  designed  into 
several  major  systems.  RCA  also  con¬ 
tinues  to  supply  rad-hard  devices  to 
selected  military/aerospace  programs. 

Rockwell 

Rockwell  has  been  a  major  supplier  of 
semicustom  military  devices  to  such 
agencies  as  SAMSO  and  NASA  in  the 
area  of  secured  communication  systems 
as  well  as  reconnaissance  and  spy-in- 
the-sky  satellite  systems.  Rockwell 
has  bid  Phase  O  of  the  VHSIC  program 
and  has  made  long-term  major  invest¬ 
ments  in  silicon  and  gallium  arsenide 
technologies.  Rockwell  has  done  an 
effective  job  in  supporting  and  servic¬ 
ing  the  program  management  offices  at 
various  DoD  agencies. 

Captive  Military/Aerospace  IC  In- 
House  Facilities 

Over  the  last  decade,  the  military 
contractors  have  been  unable  to  use 
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captive  IC  facilities  effectively  for  pro¬ 
duction  of  ground-based  computer  eq¬ 
uipment.  For  the  next  decade,  there 
is  growing  realization  that  the  integra 
tion  level  of  military  components  will 
have  to  be  raise<^  significantly.  .  .  to 
take  advantage  of  the  enhanced  reli¬ 
ability  and  lower  costs  associated  with 
V LSI.  The  VHSIC  program  is  aimed  at 
this  objective,  but  contractors  will  con¬ 
tinue  to  use  a  number  of  approaches,  in¬ 
cluding  standard  design,  custom  design 
with  outside  manufacture,  and  in-house 
design  and  manufacture.  The  sophis¬ 
tication  level  of  IC  processing  is  ex¬ 
pected  to  be  very  high,  encouraging  in- 
house  design  coupled  with  outside  pro¬ 
cessing  by  a  qualified  volume  producer. 
The  status  of  various  process  tech¬ 
nologies  at  selected  military/aerospace 
system  houses  is  reviewed  in  Table  5-8. 

PRIME  VHSIC  PROGRAM  GOALS 
PROVIDE  THE  DoD  IC  WITH: 


•  REAL-TIME,  HIGH  SPEED,  MILITARY  SIGNAL 
PROCESSING  CAPABILITIES 

•  MILITARY  SYSTEM  ARCHITECTURE  AND  IC  DESIGN. 
THIS  REDUCES  COST/TIME  OF  IC  ACQUISITION 


SUCCESSFUL  COMPLETION  OF  THE 
VHSIC  PROGRAM  WILL  RESULT  IN: 


•  DoD  BEING  ABLE  TO  EXTEND  ITS  TECHNOLOGICAL 
LEADERSHIP  IN  MILITARY  ELECTRONICS 

•  REDUCE  LIFE  CYCLE  COSTS 

•  INSERTION  OF  NEW  VHSIC  TECHNOLOGY  INTO 
MILITARY  SYSTEMS 

REASON:  THE  PROGRAM  IS  SYSTEM  DRIVEN 
THE  VHSIC  DEVICES  ARE 
AVAILABLE 
AFFORDABLE 
EASY-TO-USE 


PHASE  0 

CONTRACT  PROGRAM  ELEMENTS 


•  MILITARY  ELECTRONICS  SYSTEMS  ARCHITECTURE 
CAD  SYSTEMS  AND  METHODS 

TEST  TECHNOLOGY  AND  TEST  SYSTEMS 

•  ADVANCE  IC  TECHNOLOGY  DEVELOPMENT  PROCESSES 
PROCESS  EQUIPMENT 


PHASE  0  BIDDERS  WHO  WERE  NOT 
AWARDED  PHASE  0  CONTRACTS 


SYSTEM 

MANUFACTURERS . 


U.S.  SEMICONDUCTOR 
MANUFACTURERS 


TECHNOLOGY 


HARRIS  SEMICONDUCTOR  CMOS.  SOS/CMOS 


BOEINGIUNIV.  OF  UTAH  GENERAL  INSTRUMENT 

BELL  LABS  WESTERN  ELECTRIC 


NMOS.  BIPOLAR 


CMOS, 

N-CHANNEL  MOS 


MILITARY  IC  CHIP  DESIGNS  TO  INCLUDE  ON-CHIP 
TESTING  AND  FAULT  TOLERANT  CONCEPTS 

IC  TECHNOLOGY  SUITABLE  FOR  MILITARY 
QUALIFICATION 


JAN  3SSI0  SAMS  DIGITAL  IC  MARKET 
1»7«  to  1BB1 
CLASS  B 
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THE  VHSIC  PROGRAM 
IS  A  CATALYTIC  EFFORT  THAT  WILL 
BENEFIT  INDUSTRY  BY 
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INCREASING  MILITARY  SALES 
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•  STIMULATING  INDUSTRIAL  INNOVATION 

•  ADVANCING  THE  GENERAL  IC  TECHNOLOGY  THROUGH 
PRODUCT  AND  EQUIPMENT  ADVANCEMENTS 
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TEAM  MEMBERS  WINNING  PHASE  0  CONTRACTS 

CONTRACTS  AWARDED  MARCH  7.  1980 
STARTED  APRIL  12.  1980— COMPLETED  JANUARY.  1981 


SYSTEM 

MANUFACTURERS 

U.S. 

SEMICONDUCTOR 

MANUFACTURERS 

TECHNOLOGY 

DoO  AGENCY 
ASSIGNMENT 

Tl 

Tl 

N-CHANNEL  MOS 
l»L 

AIRFORCE 
(WPAF  DAYTON) 

RAYTHEON 

FAIRCHILD 

ECLCCO 

AIRFORCE 
(WPAF  DAYTON) 

HONEYWELL/3M 

BIPOLAR 

AIRFORCE 
(WPAF  DAYTON) 

IBM 

- 

N  CHANNEL  MOS 
BIPOLAR 

NAVY 

(WRL  WASHINGTON) 

TRW/UNIVAOQCA 

MOTOROLA 

BIPOLAR 

TRIPLE  DIFFUSED 
CMOS 

NAVY 

(WRL  WASHINGTON) 

WESTINQHOUSE 

NATIONAL 

SEMICONDUCTOR 

CMOS.  BIPOLAR 

NAVY 

(WRL  WASHINGTON) 

HUGHES 

SIQNETICS 

1*L.  CMOS 

ARMY 

(FT  MONMOUTH.  N.J.) 

ROCKWELL/SANOERS 

ROCKWELL 

CMOS.  CMOS/SOS 
N-CHANNEL  MOS 

ARMY 

(FT.  MONMOUTH,  N.J.) 

Norr 


AlHfOHCE  VHStC  PROURAM  DIRECTOR 


NAVY  VHStC  PROGRAM  DIRECTOR 
ARMY  VHStC  PROGRAM  DIRECTOR 


MR  JOHN  ILASINOAME 
AVIONICS  CA9  ELECTRONIC 
TECHNOLOGY  OlV 
MR.  NATHAN  BUTLER 
NAVAL  ELECTRONICS  COMMAND 
OR  CLARE  THORNTON 
ELECTRONICS  U8 


VHSIC  DESIGN  RULES 


_ NEAR  TERM,  13  MICRON  DESIGN  RULES _ 

1.  GREATER  THAN  30  MHc  CLOCK  RATE 

3  STATIC  RAM  CHIPS  (84K  eiT  MINIMUM) 

3.  AOOirrONAL  CHIP  TYPES  SUCH  AS  PLAf.  CGAs,  A/0  CONVERTERS.  MULTIPLIERS. 
ETC.  (FINAL  SET  COMPATIBLE  WITH  IDENTIFIED  SYSTEM  DEMONSTRATIONS) 

4  FIGURE  OF  MERIT  GREATER  THAN  lO  '/CM'  NUMBER  OF  EQUIVALENT  GATES  TIMES 
CLOCK  RATE 

5.  CHIPS  CAPABLE  OF  MIL  SPEC  PERFORMANCE 

6.  CHIPS  TOLERANT  OF  MAN-SURVIVABLE  RADIATION 

7.  INCLUSION  OF  APPROPRIATE  FAULT-TOLERANCE  AND  BUILT-IN  TEST  DESIGN 

8.  DEDICATED  LINE  PRODUCTION  CAPABILITY 

9.  CHIP  SETS  FOR  IDENTIFIED  SYSTEM  DEMONSTRATIONS 

10.  APPROPRIATE  TEST  CHIPS  TO  DEMONSTRATE  MULTI  LEVEL  METALIZATION  AND 
OTHER  FEATURES  NECESSARY  FOR  EVENTUAL  SUBMICRON  CHIP  DEVELOPMENT. 


_ LONG  TERM.  SUBMICflON  (0.5  0-6  MICRON)  DESIGN  RULES _ 

1.  GREATER  THAN  100  MH2  CLOCK  RATE 

2.  STATIC  RAM  CHIPS  (%  MBIT  MINIMUM) 

3  FIGURE  OF  MERIT  OF  ABOUT  10  ‘ICM*  (NUMBER  OF  EQUIVALENT  GATES  TIMES 
CLOCK  RATE) 

4.  CHIPS  CAPABLE  OF  MIL  SPEC  PERFORMANCE 

$.  FAILURE  RATE  EQUAL  OR  LESS  THAN  0.1%  PER  1000  HOURS  AT  13S*C  AMBIENT- 

6.  FAULT-TOLERANT/BUILT-IN  TEST  ARCHITECTURES 

7.  DEDICATED  LINE  PRODUCTION  CAPABILITY 

6.  CHIP  SETS  FOR  IDENTIFIED  SYSTEM  DEMONSTRATIONS 


U.S.  SEMICONDUCTOR  MANUFACTURERS  SUPPORT 
OF  OoO/AEROSPACE  PROGRAMS  AND  TECHNOLOGY 
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IN  HOUSE  TECHNOLOGIES 
LOCATED  AT  MILITARY/AEROSPACE 
_ SYSTEM  HOUSES 
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MILITARY/AEROSPACE  SUPPLIERS 


A  SELECTED  REVIEW  OF 
CANDIDATE  SYSTEMS  FOR  PHASE  0 
VHSIC  PROGRAM  DEFINITION  PHASE 


DoD  AGENCY 

PROGRAM/SYSTEM 

U.S.  ARMY 

TACTICAL  INTEGRATED  EW  WEAPONS  SIGNAL 

PROCESSOR  (PARAMETER  MEASUREMENT  UNIT, 
MODULAR  ADAPTIVE  SIGNAL  SORTER,  ETC.) 

FIRE  AND  FORGET  MISSILE  ELECTRONICS  (RF  AC 
TIVE/PASSIVE  AND/OR  EO  PROCESSOR) 

ADVANCED  TACTICAL  DATA/VOICE  DISTRIBUTION 
AND  NAVIGATION  PROCESSOR  (ADDS  PLUS) 

U.S.  ARMY 

BACK-PACK  COMMUNICATIONS 

U.S.  NAVY 

MICRO  VECTOR  PROCESSOR  (MVP) 

F-14  CILOP  MULTISENSOR  PROCESSOR 

(TENTATIVE 

E-2C  AEW  RADAR 

SYSTEMS) 

FIGHTERIATTACK  INTEGRATED  RADAR  (FAIR) 
TACTICAL  INFORMATION  EXCHANGE  SYSTEM 
(TIES) 

U.S.  NAVY 

ARRAY  VECTOR  PROCESSOR  (ACOUSTIC) 

AYK14  UYK-20 

RADAR  SIGNAL  PROCESSOR 

IMAGE  PROCESSOR  (SAR) 

BEANFORMER  (SOSUS) 

NATO  ID  SYSTEM 

ANTIJAM  COMMUNICATIONS  MODEM 

32  BIT  GENERAL  PURPOSE  PROCESSOR 

U.S.  AIR  FORCE 

PROGRAMMABLE  SIGNAL  PROCESSORS  FOR  AD¬ 
VANCED  AIRBORNE  ALL-WEATHER  TACTICAL 

STRIKE 

COMMUNICATIONS  SIGNAL  PROCESSOR  FOR 

JTIDS 

ATTACK  RADAR 

ODOOOobooboo I 


SUPPUER  OVERVIEW  OF  SPACE 
PARTS 

MICROELECTRONIC 

PROCUREMENT 

Dick  Lambert 

SIGNETICS  CORPORATION 
Chairman,  JEDEC  JC-13.  2 

Ladies  and  gentlemen;  when  Jim 
Wiesner  called  to  ask  that  someone 
from  my  Government  Liaison  Comm¬ 
ittee  speak  to  this  group  on  space  parts 
procurement  issues,  I  knew  this  some¬ 
one  would  be  placed  in  a  difficult,  no 
win  position.  Our  JEDEC  efforts  have 
been  directed  toward  the  solution  of 
technical  issues  and,  unfortunately, 
there  is  no  legal  means  to  coordinate 
supplier  positions  on  subjects  that  re¬ 
late  to  procurements.  This  single  area 
has  been  most  overlooked  in  the  recent 
developmental  years,  and  is  a  very 
significant  contributor  to  today's  pro¬ 
blems.  Having  picked  the  short  straw 
in  my  committee,  I  hope  today  to  pre¬ 
sent  the  suppliers  view  of  spaceparts 
procurement  without,  hopefully,  show¬ 
ing  too  much  of  my  Signetics'  bias,  or 
otherwise  being  placed  in  a  compromis¬ 
ing  position  with  my  supplier  counter¬ 
parts.  I  cannot,  unfortunately,  speak 
as  a  single  representative  for  my  in¬ 
dustry  on  these  issues--and  with  that 
disclaimer  out  of  the  way.  I'll  proceed. 

In  support  of  the  stated  theme  of  this 


workshop,  I've  segregated  my  com¬ 
ments  to  the  five  bullits  on  my  first 
foil.  I'll  be  commenting  on  the  market, 
small  volume  buys,  coordinated  pro¬ 
curement,  the  factors  that  motivate  or 
otherwise  demotivate  suppliers,  and 
fianlly.  I'll  wrap  up  with  a  few  sugg¬ 
estions  on  how  to  go  forward  with  this 
program.  If  you'll  hold  your  comments 
and  questions,  there'll  be  time  for 
these  at  the  end  of  the  presentation. 

Our  view  of  the  spaceparts  market  is 
one  of  the  mixed  emotion.  The  indus¬ 
try  data  available  on  the  market  indi¬ 
cates  a  larger  share  of  the  overall  DOD 
market  going  to  space  over  the  next  10 
years.  Unfortunately,  we  lack  specif¬ 
ics.  The  major  programs--the  atten¬ 
tion  getters- -have  not  settled  into  a 
regular,  predictable  procurement  cycle. 
Other  smaller  programs  pop  up  sporad¬ 
ically,  without  the  commitment  for 
follow-on  requirements,  specifically, 
NASA  requirements  have  been  masked 
by  the  inordinate  Shuttle  proccupation 
in  recent  years --and  the  bulk  of  that 
programs  needs  are  in  Class  B,  B+  or 
Qua  si- A. 

Technical  needs  are  fragmented.  We 
continue  to  support  the  needs  of  the  old¬ 
er  programs  such  as  85MO,  MIL-M- 
0038510,  and  P-95.  The  technical  con¬ 
tent  of  these  requirements- -in  com¬ 
parison  to  today's  Class  S  or  even  with 
each  other--do  nothing  to  promote  stan¬ 
dardization,  The  myriad  of  other  pro- 
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gram  oriented,  SCD  based  require¬ 
ments,  cause  absolute  chaos  in  the 
supplier  community. 

Major  programs  dominate  and  in  them¬ 
selves  create  residuals- -these  are 
over- run  inventories.  This  would 
normally  be  good  for  all  if  only  the  re¬ 
quirements  were  identical.  Most  often 
they  are  not,  and  the  user  is  placed  in 
a  position  of  risk  vs.  availability. 

The  next  graph  illustrates  our  view  of 
the  spaceparts  electronic  growth  in  the 
next  decade,  as  compared  with  the  over¬ 
all  DOD  electronic  growth.  I  apologize 
for  the  normalization,  but  the  numbers 
are  not  only  controversial,  but  include 
far  more  than  simple  component  con¬ 
tributions.  Our  assumption,  of  course, 
is  that  a  form  of  standardization  has 
taken  place  in  this  initial  phase  so  that 
this  growth  is  supplemented  by  the  com¬ 
bining  of  similar  requirements.  The 
next  foil  is  an  optimistic  view  of  what 
we'd  like  to  see  in  the  market's  trans¬ 
ition  to  JAN  Class  S  procurement.  Let 
me  restate  that- -this  illustrates  what 
we  expect  to  see  in  the  market  over  the 
next  few  years.  We  would  be  very  dis¬ 
appointed  to  see  anything  less  ambitious. 
JAN  Class  S  standardization  is  manda¬ 
tory  if  the  system  of  space  component 
procurement  is  to  succeed. 

You  can  see  that  we  expect  to  reach  50% 
procurement  standardization  by  1982.  I 
believe  the  suppliers  will  provide  the 
primary  motivation  to  meet  this  objec¬ 


tive.  Non- standard  procurement  to 
custom  SCD  requirements  will  become 
extremely  difficult,  if  not  impossible. 

In  addition,  an  increasing  segment  of 
the  diminishing  non-JAN/SCD  procure¬ 
ment  will  consist  of  part  types  with  no 
existing  slash  sheets,  but  will  be  pro¬ 
cessed  to  an  identical  38510  flow  as  JAN 
Class  S.  Custom  flows  with  all  the  ass¬ 
ociated  bells  and  whistles  will  become  a 
rarity.  Spec  negotiation  will  become 
absolutely  non  existant. 

I've  listed  the  problems  of  small  volume 
procurement  on  the  next  foil,  and  I  ex¬ 
pect  a  lot  of  head  nodding  on  this  one. 
The  problems  are  somewhat  obvious, 
and  somewhat  analgous  to  the  situation 
with  Class  B  microcircuits  in  the  1972- 
1975  time  frame.  It  is  extremely  diff¬ 
icult  to  get  the  ball  rolling,  but  once  it 
is,  it's  all  down  hill. 

At  the  moment  JAN  Class  S  microcir¬ 
cuits  are  simply  not  available.  A  very 
few  users  have  had  the  inclination  to  buy 
straight  JAN  Class  S  and  fewer  sup¬ 
pliers  have  committed  to  an  inventory 
plan  of  any  kind.  Those  bloodied  sup¬ 
pliers  who  have  survived  the  certifi¬ 
cation  process  realize  now  that  the  de¬ 
mand  on  resources  to  support  the  pro¬ 
gram  is  significant.  Most  are  unwilling 
to  go  further  by  initiation  qualification 
testing  without  user  procurement. 

Lack  of  JAN  availability  promotes  and 
somehow  endorses  non-standard  pro¬ 
curement.  SCD  requirements  usually 
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lack  the  MIL  Spec  due  to  the  volatile 
nature  of  the  evolving  requirements. 

Out  of  date  SCD's  eventually  turn  into 
non-standard  requirements  thereby 
feeding  fuel  to  the  problem. 

Initial  procurements  lare  small,  high 
risk,  and  are  fixed  priced  contracts. 
Suppliers  are  producing  first  article 
part  types  and  face  start  up  problems 
associated  with  the  most  difficult  class 
of  product  available.  Little  or  no  in¬ 
ventories  exist  and,  as  a  result,  lead 
times  are  unbearably  long  and  very 
uncertain. 

Coordinated  procurement  at  this  point 
doesn't  do  much  for  the  supplier.  The 
obvious  benefits  associated  with  larger 
order  quantities  are  countered  by  a  few 
technical  and  administrative  difficulties. 
Specifically,  non-standard  requirements 
in  the  electrical  characteristics  or  the 
process  flows  detract  from  the  coordin¬ 
ation  benefits.  Procurement  delays  re¬ 
sult  from  the  basic  communication  pro¬ 
blems  existing  between  the  prime  and 
sub,  the  problems  involved  in  device 
selection,  and  the  difficulties  associated 
with  pricing  non-standard  requirements. 
True  coordination,  of  course,  would  be 
nice--but  a  dedication  must  first  be 
made  to  uncompromised  standardiza¬ 
tion.  When  all  parties  agree  on  straight 
JAN  Class  S,  coordinated  procurement 
issues  will  disappear. 

Now  I'd  like  to  go  into  the  motivation 
issues- -these  can  best  be  illustrated  by 


going  through  a  bit  of  recent  history. 
Back  in  the  mid  70' s  many  manufac¬ 
turer's  were  participating  in  the  NASA, 
Class  A  certification  program.  True 
Class  A  processed  parts  were  non- 
existant-- every  program  had  its  unique 
set  of  characteristics  and  flows- -JAN 
standardization  simply  wasn't  a  serious 
issue.  Yet  manufacturers  seemed  blind 
to  these  shortcomings,  and  eagerly  pur¬ 
sued  more  and  more  certifications 
The  issue  was  that  of  being  in  a  ready 
position  should  an  opportunity  arise. 

It  was  nice,  also,  to  paper  the  lobby 
with  the  certifications- -nice  copy  to 
show  off  to  your  customers. 

Just  as  the  ROI  reality  was  dawning  in 
1976,  SAMSO  and  Aerospace  launched 
their  successful  effort  to  coordinate 
suppliers  and  users  on  "The  New"  Class 
S  set  of  requirements.  The  lure  of  po¬ 
tential  standardization  revitalized  the 
spirit  of  the  suppliers.  In  1977,  the 
first  set  of  Class  S  MIL-Specs  were  on 
the  streets  and,  in  December,  SAMSO/ 
Aerospace  had  a  kick-off  meeting  to 
motivate  all  of  the  potential  suppliers. 
GPS  and  lUS  loomed  on  the  horizon  as  a 
great  carrot  in  the  sky.  In  1978,  the 
benefits  of  standardization  seemed  to 
temper  the  risk  of  lost  investment-also, 
the  initial  certifications  were  then  in 
progress.  Efforts  to  fine  tune  the  specs 
were  continuing  with  excellent  cooper¬ 
ation  between  NASA,  SAMSO,  Aero¬ 
space  and  JEDEC.  Despite  a  lengthy 
certification  learning  curve,  the  first 
certifications  were  issued  in  early  1979. 


238 


Manufacturers  waited  breathlessly  for 
the  onslaught  of  orders  after  the  fii 
Class  S  parts  were  listed  on  the  QPIj. 

1980  holds  the  hope  for  the  final  con¬ 
version  of  the  major  programs  to  JAN 
Class  S.  Every  indicator  points  to  a 
successful  climax  to  this  long,  pain¬ 
full  process.  This  is  gratifying. 

Unfortunately,  there  remains  a  few  de¬ 
motivators  which  influence  today's  pro¬ 
curement  situation.  Agency  interactive 
delays  have  caused  valuable  time  lost  in 
the  certification/qualification  process, 
and  this  causes  delays  in  the  introduc¬ 
tion  of  legitimate  QPL  part  types. 
Granted,  improvement  has  been  dra¬ 
matic  since  the  first  struggling  days, 
but,  to  an  impatient  supplier,  this 
factor  represents  a  disappointment. 

Options,  whether  contract  imposed  or 
allowable  in  MIL-Spec,  demotivate  a 
supplier  from  investing  in  inventories 
that  may  turn  out  non-saleable  because 
of  the  existance,  or  lack  of,  a  given 
option.  This  type  of  procurement  ap¬ 
proach  promotes  non-standardization 
simply  becuase  of  the  "personalization" 
of  the  resultant  lots.  Residuals  from 
these  lots  are  construed  by  other  users 
as  S  +,  S-,  or  simply  unusable  because 
of  the  simple  uncertainty  of  the  matter. 
A  single  non-standard  option  breeds 
curiousity  about  possible  other  non¬ 
standard  slip-in's  such  that  hours  are 
invested  by  both  user  and  supplier  in 
cleansing  the  lots. 


The  technical  MIL-Spec  requirements 
are  to  say  the  least  volatile.  I  might 
add  that  predominantly  the  suppliers 
have  been  pressing  for  the  majority  of 
these  changes  based  on  purely  textbook 
reviews.  The  demotivating  factor, 
oddly  enough,  comes  from  the  effect 
felt  in  having  the  desired  changes  made 
to  MIL-Spec,  only  to  find  that  residuals 
suddenly  become  non-standard.  Since 
the  entire  Class  S  Spec  system  is  large¬ 
ly  untested,  one  could  expect  a  host  of 
subtle  problems  to  surface  during  the 
first  few  production  runs. 

We  believe  the  largest  demotivator  is 
in  the  attitude  of  the  space  community 
users.  We  sense  a  user  defensiveness 
which  causes  large  expenditures  of 
effort  to  avoid  procurement  of  JAN 
Class  S.  This  defensiveness  is  on  the 
rationale  experienced  by  the  suppliers 
during  the  earlier  days  of  Class  B,  as 
I  mentioned  earlier.  Cost-lead  time- 
availability-  and  the  lack  of  second 
sources.  If  you  are  looking  for  a  rea¬ 
son  not  to  buy  microcircuits-- JAN 
Class  S--go  ahead  and  pick  one.  Per¬ 
haps  more  viable  a  cause,  users  cling 
to  CSI-Customer  Source  I nspection-to 
validate  the  screening  of  the  devices 
throughout  the  process.  This  has  been 
ruled  to  be  a  "personalized"  require¬ 
ment  and  that  government  source  shall 
suffice.  The  large  user  resistance  is  a 
significant  negative  factor  to  the  sup¬ 
plier.  A  less  significant  factor  is  the 
requirement  of  the  classic  space  user 
for  baseline  control.  Admittedly,  it  is 


difficult  to  abandon,  to  another  party, 
responsibility  for  the  control  of  the 
products. 

Last,  but  certainly  not  least,  our  pat¬ 
ience  wanes  as  the  ROI  stays  just  be¬ 
yond  our  grasp,  and  Class  B  demands 
on  our  resources  and  capacity  continue 
to  grow  exponentially.  Revenue,  or 
lack  of  it,  has  a  strangely  dramatic 
effect  on  supplier  motivation. 

What,  then,  is  at  least  one  suppliers 
recommendations  to  press  forward  with 
the  momentum  of  the  past? 

Re-examine  the  QPL  rules.  If  the  lack 
of  a  QPL  source  causes  the  user  to 
write  a  non-standard  SCD  and  procure 
from  a  non-certified  supplier,  why  not 
instead  allow  a  certified  supplier  to 
supply  JAN  Class  S--and  demonstrate 
its'  ability  to  qualify  the  part  prior  to 
initial  deliveries?  The  user  assumes 
risk  in  both  situations,  but  surely  the 
risk  is  less  in  the  latter  case--and, 
it  would  promote  the  QPL  in  the  process. 

Place  a  moritorium  on  all  new  require¬ 
ments.  Why  not  allow  a  little  time  to 
shake  out  the  system  with  a  few  orders 
before  making  more  non-standard  resi¬ 
duals? 

Resolve  the  Class  S  options.  Resolve 
yourselves  to  not  being  able  to  get  more 
than  one  version.  Sit  down  and  work 
out  a  compromise  acceptable  to  all 
parties. 


Commit  to  JAN  Class  S.  I  mean  stand 
up  and  make  an  uncompromised  comm¬ 
ittment  to  this  program-and  it  will  work. 
There  can  be  no  conscious  thought  of 
alternatives  unless  it  is  absolutely  nec¬ 
essary. 

Forget  the  bells  and  whistles.  Accept 
the  fact  that  Class  S  is  an  order  of  mag- 
natude  more  reliable  than  Class  B,  and 
that  your  little  whiz-bang  requirement 
will  exert  a  small  influence  on  that 
fa  ctor,  if  at  all. 

We  feel  quite  strongly  on  these  as  being 
somewhat  essential  to  the  health  of  the 
system.  One  of  the  immerging  strat¬ 
egies,  which  I  believe  to  serve  to  pro¬ 
mote  the  procurement  issues,  is  that  of 
contractor  triggered  stocking.  A  simple 
system--the  first  user  triggers  the  sys¬ 
tem  through  his  initial  buy,  and  a  plann¬ 
ed  stocking  program  is  augmented  in 
both  die  and  finished  forms.  On  sub¬ 
sequent  buys,  these  inventory  levels  are 
examined  and  adjusted  according  to  es¬ 
tablished  run  rates.  Small  volume  needs 
are  served  through  a  planned  approach  to 
to  residuals.  Over-builds  allow  lot  test 
charges  to  be  amortized  into  larger 
quantities  of  the  pieceparts  such  that 
these  charges  are  less  painfull,  and 
such  that  the  first  buyer  isn't  need¬ 
lessly  punished  by  absorbing  all  the 
test  charges.  We  think  it's  a  good  sys- 
tem--one  ready  to  be  tested. 

Lastly,  something  should  be  done  to 
provide  better  forecasting  of  program 


material  needs.  Lead  time  can  hardly 
improve  if  forecasts  continue  to  be 
uncertain  in  both  quantities  and  time. 


SPACE  MICROELECTRONIC  MARKET 


My  industry  is  to  a  large  degree  en¬ 
thusiastic  about  the  JAN  Class  S  pro¬ 
gram.  We're  willing  to  work  with  you 
to  satisfy  your  fears,  concerns,  and 
heart-burn.  Let's  make  this  effort 
rewarding  to  all! 

Thank  You. 


SUPPLIER  OVERVIEW  OF  SPACEPART 
MICROELECTRONICS  PROCUREMENT 


•  SPACE  MICROELECTRONIC  MARKET 

•  SMALL  VOLUME  PROCUREMENT 

•  COORDINATED  PROCUREMENT 

•  SUPPLIER  MOTIVATION 

•  RECOMMENDATIONS 


JAN  CLASS  S  PROCUREMENT  TRANSITION 
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•  DOD  PROCUREMENT  PROJECTED  GROIfTH 

-TOTAL  ELECTRONICS:  SI  ANNUALLY 
-SPACE  ELECTRONICS:  13Z  ANNUALLY 

•  NASA  NEEDS  UNCERTAIN 

•  TECHNICAL  REQUIREMENTS  FRAGMENTED 

•  MAJOR  PROGRAMS  DOMINATION 

•  RESIDUALS 


SMALL  VOLUME  PROCUREMENT  PROBLEMS 
f  RESOURCE  DEMANDING 

0  NON-STANDARD  REQUIRBCNTS 

0  HIGH  RISK/FIXED  PRICE  CONTRACTS 

0  SMALL  INVENTORIES 

0  LONG  LEAD  TIMES 
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SMALL  VOLUME  PROCUREMENT 


Dale  M.  Cole 

General  Electric  Company 
Aircraft  Equipment  Division,  Utica,  NY 


A  major  challenge  facing  the  system  manufacturers  of  space-oriented 
equipment  is  assuring  that  the  program's  schedules,  cost,  and  quality  goals 
are  obtained  on  the  ever-increasing  small  volume  lots  of  components.  This 
problem  is  amplified  by  the  fact  that  the  military  part  procurement  over 
the  years  has  become  less  and  less  of  the  total  market  place.  One  has  only 
to  look  at  the  history  of  military  procurement  of  semiconductors  over  the 
last  decade  and  into  the  mid-eighties  to  realize  that  no  longer  is  the 
military  industry  talking  from  strength  to  the  semiconductor  industry.  Fig¬ 
ure  1  shows  that  this  trend  is  definitely  downward  and  it  is  predicted  to 
continue  to  go  down  to  as  low  as  3  percent  in  the  1985  time  frame.  Of 
course,  within  the  military  3  percent,  the  space  industry  procurement  is 
estimated  to  be  less  than  0.3  percent  of  the  total. 


YEAR 

Figure  1.  Military/Aerospace  Market  Trend  -  U.S.  Semiconductor  Industry 
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The  effect  of  this  market  share  on  a  given  device  type  was  emphasized 
in  a  recent  microprocessor  experience  study  performed  by  GE  Aerospace  Elec¬ 
tronic  Systems  Department  (GE/AESD)  for  NASA/MSFC  (Contract  NAS8-33142). 
Evidence  of  one  of  the  major  causes  (automotive  industry)  of  the  erosion 
was  clearly  Indicated  in  this  study  as  Figure  2  shows  the  impact  that  the 
automotive  Industry  is  having  on  the  1802  microprocessor  market  in  the 
1980-81  time  frame.  Prior  to  1979,  the  only  use  in  the  automotive  industry 
was  small  prototype  quantities  to  prove  the  feasibility  of  using  the  1800 
family  in  an  electronic  spark  control  system.  This  device  proved  to  be  an 
excellent  choice  in  this  environment  due  to  its  noise  immunity  capability 
and  lower  {)ower  consumption.  Therefore,  pilot  runs  (''-100, 000)  of  this  sys¬ 
tem  were  manufactured  in  the  1979  models  and  full  production  ('v-500,000) 
will  take  place  on  the  1980  models. 

1100  h  ^ - - 


1978  1979  1980  1981 

YEAR 


Figure  2.  1802  Microprocessor  Volume  Market  Forecast 


Listed  below  are  some  of  the  major  causes  of  small  lot  procurement: 

•  Government  funding  policy  and  schedule  requirements 

•  Use  of  state-of-the-art  devices 

•  Lack  of  standardization 

•  Special  requirements. 
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Government  Funding  Policy  and  Schedule  Requirements 


As  all  of  us  know,  the  government  overall  funding  policy  is  at  best 
an  on  and  off  again  thing,  depending  on  Congress's  mood,  the  President's 
and  DoD's  requests.  On  top  of  this,  we  are  expected  to  deliver  at  the 
maximum  hundred  of  systems  spaced  over  a  multiple  year  procurement.  All 
of  this  makes  It  next  to  Impossible  for  the  aerospace  manufecturers  to 
forecast  their  component  requirements  for  more  than  one  year  with  any 
validity.  However,  It  Is  surprising  how  accurately  (+5Z)  the  consumer/ 
Industrial  complex  can  predict  their  usage  of  millions  of  parts  year  after 
year. 

Use  of  State-of-the-Art  Devices 


As  the  density  and  complexity  of  microcircuits  continues  to  Increase 
and  more  and  more  of  the  system  function  Is  designed  onto  a  chip,  the  quan- 
r  tlty  procurement  of  any  given  device  becomes  significantly  less.  Figure  3 

f  emphasizes  this  for  a  minicomputer.  Where  previously  in  SSI  and  MSI  com¬ 

plexity  It  took  16  devices  of  five  different  types,  now  one  LSI  device  is 
the  complete  function.  Therefore,  for  a  normal  military  fifty  to  a  hundred 
system  buy,  the  requirements  would  be  50  to  100  LSI  devices  versus  800  to 
1600  SSI  and  MSI  devices,  dictating  a  small  volume  procurement. 


NCMMOCEttM 


Figure  3.  Use  of  State-of-the-Art  Devices 
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As  complexity  Increases,  this  trend  will  continue,  causing  many  of 
the  small  volume  procurements  to  be  placed  on  the  state-of-the-art  devices 
(e.g.,  microprocessors,  FPLAs,  etc.)  that  are  just  leaving  the  learning 
curve.  In  addition,  the  aerospace  community  timing  of  production  procure¬ 
ment  on  state-of-the-art  devices  usually  coincides  with  the  peak  demand 
of  the  consumer/ Industrial  community.  An  excellent  example  of  this  Is  the 
severe  shortage  of  the  16K  dynamic  RAM  (4116)  that  the  consumer /Indus trial 
community  Is  experiencing  due  to  the  computer  Industry  needs;  at  the  same 
time,  the  military  complex  Is  finding  It  being  designed  Into  almost  all 
new  memory  designs.  Figure  4  emphasizes  graphically  this  problem. 
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Figure  4.  Typical  Time  Phasing  of  New  Device  Designs 
Into  the  Market  Place 

Lack  of  Standardization 

Without  a  standardization  program  In  the  front  end  of  a  new  design, 
proliferation  of  different  parts  types  Is  sure  to  take  place,  thereby  In¬ 
creasing  the  number  of  small  lot  procurements  required  for  that  design. 

Special  Requirement 

Every  time  an  application  requires  a  standard  part  with  special  re¬ 
quirement  (l.e.,  2N2222  with  a  BVceo  ”  60  V) ,  It  Inadvertently  necessitates 
a  small  lot  procurement. 

Problems 


Once  a  system  manufacturer  Is  forced  Into  a  small  volume  procurement, 
the  problems  he  faces  are  many.  This  paper  deals  only  with  the  major  ones 
listed  below. 


Small  Volume  Procurement  Problems 


•  Schedule 

•  Cost 

•  Supplier  Follow-through 

•  Lot-to-Lot  Quality 

•  Qualification 

•  No  Bids 

Solutions 


Because  the  small  lot  procurement  problems  have  been  with  us  for  some 
time,  throughout  the  Industry  some  solutions  to  these  problems  have  been 
Implemented  or  suggested.  Of  course,  there  are  pros  and  cons  to  .many  of 
these  solutions  and  therefore  It  is  the  author’s  intent  at  this  time  to 
bring  these  solutions  forward  so  that  they  may  ferment  in  your  mind  until 
the  panel  discussion  on  this  subject  is  held  this  afternoon.  If  these 
problems  and  solutions  presented  here  can  be  used  to  form  a  basis  for  in- 
depth  discussions  on  this  subject  this  afternoon,  then  this  paper  has  ful¬ 
filled  its  purpose.  The  problems  with  the  corresponding  solutions  are 


listed  below. 

Problem 

Schedule  and  Cost 

Solutions 

• 

Pool  Procurement 

• 

Coordinated  Procurement 

• 

Common  Specification  Procurement 

• 

Minimum  Buy 

Problem 

Supplier  Follow-through 

Solutions 

• 

Resident  Inspectors 

• 

Expedite  Orders 

Problem 

Lot-to-Lot  Quality 

Solutions 

• 

Physical  Destructive  Analysis 

• 

100%  Temperature  Testing  at  Incoming 

• 

Percent  Defective  Allowed  (5%-10%) 

Problem 

Qualification 

Solutions 

• 

Mlni-Quallf Icatlon 

•  Manufacture  on  Certified  Class  "S"  or  "B"  Line 

•  Use  Supplier  "High-Rel  Equivalent" 


Problem  No  Bid 

Solutions  •  Procure  Commercial  Devices  and  Upgrade 

•  Some  Type  of  Centralized  Procurement 


Recommenda  t Ions 


In  order  to  decrease  the  impact  that  the  small  volume  procurement  has 
on  schedule,  quality,  availability  and  cost,  the  following  recommendations 
are  presented  for  consideration: 

•  The  military  Space  community  should  make  provisions  for  upgrading 
lower  level  screened  parts  (Class  "B")  for  use  In  Class  "S" 
applications. 

•  Establish  a  standardization  program  that  would  be  accepted  by  all 
military  Space  program  offices.  This  standardization  program 
should  Include  both  standard  part  types  and  standard  screening 
requirements . 

•  Microcircuit  suppliers  should  establish  a  minimum  standard  base¬ 
line  for  their  so-called  "equivalent”  devices. 


PARTS  PROGRAM  MANAGEMENT 

A  CRITICAL  PART  OF 
MISSION  ASSURANCE 

WHAT  NEXT? 

By 

Paul  Dick 

General  Electric  Space  Division 


Summary 

This  paper  presents  a  summary  of  General 
Electric  Company  Space  Division  experience 
on  high  reliability  parts  programs,  both  as  a 
user  of  these  parts  in  its  products,  and  as  an 
agent  for  other  customers  needing  such  parts 
and  related  services.  Also  presented  is  our 
experience  with  coordinated  and  integrated 
parts  programs  involving  many  users  both 
international  and  domestic,  including  a  de¬ 
scription  of  the  recently  completed  DSCS  III 
Program  Integrated  Parts  System  (PIPS). 

The  paper  then  goes  into  a  discussion  on  the 
pro's  and  con's  of  customers  funding  contrac¬ 
tors  for  maintaining  an  inventory  of  critical 
parts,  subject  to  possible  obsolescence  for 
many  production  systems  beyond  those  funded. 
Recommendations  for  further  action  are  also 
presented. 

Introduction 

The  ever  increasing  demands  for  long-life 
performance  of  satellites  and  space  systems 
continues  to  dictate  the  use  of  the  highest  re¬ 
liability  parts  available  for  the  intended  mis¬ 
sion  or  application.  Specification  integration, 
procurement,  screening  and  bum-in,  accept¬ 
ance  tests  and  results  of  other  lot  related 
evaluations  add  increasing  delivery  times  as 
well  as  increased  costs  as  the  required  level 
of  reliability  of  these  devices  gets  higher. 
Coupled  with  the  impact  of  inflation  on  mate¬ 
rial  costs  and  raising  labor  rates,  the  longer 
times  for  delivery  of  such  devices  continue  to 
drive  hardware  costs  to  higher  levels.  When 
bidding  on  new  space  programs  or  follow-ons, 
contractors  must  forecast  the  costs  for  parts 
and  the  related  labor  based  on  when  the 


schedule  dictates  they  are  needed,  which  in 
turn  defines  when  the  parts  program  starts 
and  what  phases  happen  and  when.  The  typical 
phases  of  a  contractor's  parts  program  in¬ 
clude:  requirements,  specifications,  procure¬ 
ment,  quality  monitoring,  product  acceptance, 
and  parts  control.  If  the  contractor  is  man¬ 
aging  a  coordinated  procurement  for  his 
products  and  those  of  his  subcontractors,  then 
the  various  phases  noted  above  get  more  com¬ 
plex,  involving  a  high  degree  of  integration 
between  the  prime  and  "his  partners"  on  the 
parts  program.  The  financial  advantages,  as 
relates  to  the  materials  of  an  integrated  or 
coordinated  parts  program,  are  well  known 
in  that  unit  costs  of  parts  will  be  lower  as  the 
quantity  increases  and  lot  charges  from  sev¬ 
eral  different  buyers  can  be  reduced  to  a  sin¬ 
gular  lot  charge  on  a  combined  buy.  The  lot 
charges  noted  include  those  for  SEM,  data 
packages.  Group  B  or  C  tests,  DPA,  hardness 
assurance  or  any  other  required  lot  or  special 
selection  tests.  An  added  labor  savings  will 
also  be  realized  by  elimination  of  certain  com¬ 
mon  activities  that  are  no  longer  done  by  the 
subcontractor (s);  the  prime  does  the  work  for 
the  whole  team.  These  eliminated  efforts  in¬ 
clude:  specification  integration,  order  place¬ 
ments,  quality  monitoring,  product  acceptance, 
data  books,  and  management  of  the  lot  related 
test  programs  by  several  users  of  the  same 
items.  Another  set  of  benefits  of  a  coordinated 
or  Integrated  parts  program  relate  to  Program 
Management  and  Quality  Assurance.  From  a 
program  management  basis,  greater  visability 
and  impact  appreciation  is  obtained  when  parts 
are  given  such  top-level  attention  as  results 
from  a  coordinated  or  integrated  parts  program. 
Such  treatment  minimizes  the  kinds  of  sur¬ 
prises  previously  brought  to  the  program 
office's  attention  wherein  a  certain  part  or 
parts  were:  late  in  delivery,  or  had  failed  a 
screening  program,  or  had  not  done  well  in  a 
lot  test  program  such  as  DPA,  or  a  required 
reorder  for  fallouts  or  a  complete  replacement 
lot  would  require  8  to  10  months  for  acquisi¬ 
tion.  These  problems,  in  the  form  of  an  in¬ 
tegrated  parts  program  approach,  are  treated 
quicker  and  more  effectively  whether  they  be¬ 
come  problems  of  the  prime  or  the  subcontrac¬ 
tors  or  both.  On  a  quality  assurance  basis,  the 
integrated  or  coordinated  parts  program 


assures  a  common  baseline  of  product  uni¬ 
formity  and  definition,  quality  monitoring, 
acceptance  data  and  traceability,  and  reduces 
the  overall  numbers  of  different  lots  and  their 
associated  records.  It  also  assures  a  pro¬ 
gram  of  better  communications  on  any  parts 
problems  that  may  develop  and  an  integrated 
solution  basis  with  appropriate  correction 
action  for  all  users. 

Another  concept  yet  to  be  employed  to  mini¬ 
mize  the  initial  costs  of  parts  and  their  poten¬ 
tial  downstream  delivery  delay  or  "problem" 
costs  on  a  program  involves  the  implementa¬ 
tion  of  an  appropriate  stocking  program  of 
critical,  long-lead,  or  high  technology  parts 
needed  for  forthcoming  programs.  The  ad¬ 
vantages  is  two-fold  from  such  a  concept.  The 
most  accepted  advantage  would  be  program(s) 
schedule  protection  as  the  parts  would  surely 
be  ready  for  kitting  when  needed.  On  pro¬ 
duction  programs  of  frozen  designs,  such  a 
move  could  foster  schedule  Improvements  on 
"build  when  contracted  designs. "  In  effect, 
the  long-lead  or  advanced  buys  portions  of 
some  contracts,  offers  the  contractor  a  head 
start  on  getting  critical  long-lead  items  or¬ 
dered.  These  actions  do  not  in  themselves 
serve  the  same  overall  effect  of  a  stocking 
program.  The  second  advantage  of  a  stocking 
program  is  cost  savings  realized  from  a  com¬ 
bination  of  buying  materials  in  large  quantities 
and  from  buying  now.  The  large  quantity  buys 
hopefully  for  maiQ^  programs  usage,  drives 
down  unit  costs  and  lot  charges.  The  "buy 
now  concept"  is  true  protection  against  infla¬ 
tion  and  potential  limited  future  availability  of 
a  given  part  design.  The  later  case  has  hap¬ 
pened,  especially  on  proven  mature  designs 
where  a  certain  Integrated  circuit  or  transis¬ 
tor  is  no  longer  in  production  because  of  limit¬ 
ed  usage  by  others,  replacement  by  a  more 
technological  product  of  the  producer,  or  a 
marketing  decision  of  the  producer  to  no  long¬ 
er  market  the  product  for  whatever  reason. 
The  savings  in  labor  costs  for  doing  90  per¬ 
cent  of  the  work  early  and  for  multiple  pro¬ 
grams  for  a  stocking  program  will  save  the 
increased  labor  costs  of  future  years  in  the 
areas  of  Parts  Engineering,  Procurement 
Specialists,  Quality  Assurance  Engineers, 

Test  Specialists  and  other  support  personnel. 


A  minimal  sustaining  cost  for  stock  keeping 
control  personnel  will  be  required  alter  the 
front-end  efforts  are  completed. 

An  important  technical  factor  in  making  an 
appropriate  and  effective  Parts  Stocking  Pro¬ 
gram  a  reality  is  the  establishment  and  en¬ 
forcement  of  a  Preferred  Parts  list.  Design 
Engineers  must  strive  to  select  their  parts 
from  the  list  as  these  are  the  devices  that  will 
be  stocked.  So  in  reality,  the  preferred  or, 
as  many  call  it,  the  Approved  Parts  List 
(A PL)  is  the  same  as  the  Parts  Stocking  List 
(PSL) .  It  is  anticipated  that  as  new  types  or 
technology  parts  for  future  programs  are  re¬ 
quired,  they  would  be  added  to  stock  in  ad¬ 
vance  of  first-use  requirements.  This  is  to 
say  that  hopefully  the  vast  majority  of  part 
type  requirements  for  a  new  program  will  be 
satisified  from  stock.  Production  programs 
of  frozen  designs  would  benefit  to  the  100  per¬ 
cent  mark  because  all  the  required  part  types 
would  be  in  stock. 

As  part  of  the  Parts  Stocking  Program,  the 
storekeepers  assigned  to  this  activity  would 
have  to  keep  real-time  records  on  parts  issued 
and  parts  remaining  in  stock  by  part  type, 
purchase  order,  and  lot  date  codes  so  that 
when  "minimum  reserves"  are  in  stock,  an 
Immediate  reorder  process  takes  place  to  re¬ 
plenish  the  inventory.  Minimum  reserves 
criteria  would  be  based  on  a  combination  of 
projected  usage,  time  to  replenish  an  order, 
and  best  buy  situations.  Obviously,  the  same 
rules  would  apply  to  reorders  as  initial  buys 
and  that  is  the  larger  the  purchase  quantity, 
the  smaller  the  unit  costs  and  all  the  other  lot 
related  and  labor  charges  that  go  with  the 
procurement,  testing,  acceptance,  receipt, 
rnd  stocking. 

A  very  key  factor  to  making  the  entire  scheme 
work  is  obviously  the  financial  aspects.  This 
amounts  to  where  the  front-end  money  comes 
to  finance  the  Parts  Stocking  Program,  Con¬ 
tractors  are  limited  to  how  much  of  their 
resources  their  corporate  structure  and  stock¬ 
holders  would  permit  for  such  an  effort  without 
a  high  degree  of  assurance  of  continual  pro¬ 
gram  awards  by  the  space  hardware  customer 
community.  As  most  new  programs  are  fixed 


price  competitions,  there  is  no  real  way  of 
assuring  a  high  degree  of  awards.  A  Catch  22 
results  in  that  the  lowest  cost  bidder  will  usu¬ 
ally  win  and  a  good  way  to  get  costs  down  is  to 
have  the  parts  you  need  now.  But  to  have  the 
parts  you  need  now  requires  you  to  commit  a 
large  sum  of  money,  at  least  one  to  one  and  a 
half  years  ago  for  a  program  that  was  not  then 
funded,  let  alone  awarded.  In  the  end  result, 
the  customer  will  be  p^ing  a  higher  bill  for 
his  products  because  many  potential  contrac¬ 
tors  cannot  move  out  on  a  Parts  Stocking  Pro¬ 
gram  without  front  end  financial  resources. 

The  question  is  then  raised,  "What  are  the 
benefits  of  funding  space  contractors  for  main¬ 
taining  an  inventory  of  critical  parts  (subject 
to  possible  obsolescence)  for  many  space  pro¬ 
grams  beyond  those  funded?  "  This  is  an  in¬ 
teresting  debate  subject  and  before  discussing 
the  pro's  and  con's  later  in  this  paper,  a  dis¬ 
cussion  of  General  Electric  experience  on 
Parts  Management  Programs  and  a  description 
of  the  recently  completed  DSCS  III  PIPS  effort 
(DSCS  in  Program  integrated  Parts  _System) 
follows  as  it  has  bearing  on  the  subject  matter. 

General  Electric  Parts  Management  Program 
Experience 

The  advent  of  long-life  mission  requirements 
in  1960  (two  decades  ago)  lead  General  Electric 
to  develop  various  concepts,  practices,  per¬ 
sonnel  resources,  and  facilities  for  the  spec¬ 
ification,  procurement,  quality  monitoring, 
screening  and  burn-in,  and  acceptance  of  high 
reliability  electronic  parts  and  microelectron¬ 
ics.  In  the  first  decade,  over  two  million 
parts  were  processed  and  the  recent  decade 
over  four  million  additional  parts  were 
processed,  a  doubling  process  in  just  ten 
years.  These  parts  were  used  on  many  in- 
house  programs  as  well  as  for  the  programs 
of  other  space  contractors  who  contracted 
General  Electric  to  be  its  Parts  Management 
Program  Agent.  These  parts  program  man¬ 
agement  customers  have  been  primary  inter¬ 
national  in  nature.  The  General  Electric  parts 
experience  on  U.S.  Space  Programs  is  shown 
in  Table  1.  General  Electric  e}q)erience  on 
international  programs  is  shown  in  Figure  1. 


Electric  has  been  providing  Parts  Management 
Program  services  to  many  European  and  Asian 
space  companies  and  consortium.  These  cus¬ 
tomers  realized  the  importance  of  a  coordin¬ 
ated  parts  procurement  program  from  a  stand¬ 
point  of  lower  unit  costs,  fewer  lot  charges 
and  better  quality  assurance  of  their  parts. 
They  also  had  to  go  this  route  because  most  of 
the  high  reliability  parts  technology  was  in  the 
"States"  as  they  called  the  U.S. A.  It  was  dif¬ 
ficult  for  them  to  come  stateside  for  the  peri¬ 
ods  of  time  required  for  parts  work,  so  they 
contracted  with  a  major  U.S.  Aerospace  com¬ 
pany  to  be  their  agent.  The  U.S.  agent  then 
handled  the  entire  contracted  program  for  the 
delivery  to  the  customer  of  high  reliability 
parts,  related  services,  and  data  books  defined 
in  the  contractual  work  statement.  Figure  1 
details  some  of  these  programs,  geographical 
locations,  and  quantities  of  parts  delivered.  On 
the  tabulation  is  the  Nimbus-G  Unified  Parts 
Procurement  Program  (Item  4  in  Figure  1) 
geographically  Ic  at  NASA-GSFC  on  the 

map.  This  Parts  .vianagement  Program  was 
an  effort  wherein  General  Electric  coordin¬ 
ated  the  parts  program  for  a  total  of  nine  dif¬ 
ferent  NASA-GSFC  funded  experimenters  who 
provided  payloads  on  the  Nimbus-G  spacecraft. 
The  users  were  located  throughout  the  U.S. A. 
(Ball  Brothers,  JPL,  GD-Orlando,  RCA, 
Honeywell- Lexington,  Massachusetts,  and 
University  of  Wyoming)  and  Oxford  University 
in  England,  as  well  as  NASA-GSFC  who  stock¬ 
ed  all  the  program  spares  resulting  from  the 
NUPPS  (Nimbus  Unified  Parts  Procurement 
System).  This  was  our  first  "on-shore  "  co¬ 
ordinated  parts  procurement  program,  even 
though,  as  a  prime  contractor  for  many  years, 
we  managed  our  own  parts  for  designs,  such 
as  Nimbus,  OAO,  Landsat,  Mariner,  Moon 
Based  Power  Supplies,  Viking,  BSE,  and  the 
other  programs  shown  in  Table  1.  As  it  turn¬ 
ed  out,  NASA -Goddard  had  requested  General 
Electric  to  pick  up  the  parts  management  role 
for  the  experimenters  who,  if  they  had  each 
done  singularly  what  they  proposed,  would 
have  run  the  costs  of  the  same  parts  for  their 
designs  considerably  higher.  Tables  2A  and 
2B  show  a  comparison  of  the  material  cost 
savings  and  total  savings  on  the  Nimbus-G 
Unified  Parts  Procurement  Program. 


As  noted  in  Figure  1,  since  1973  General 


The  next  section  of  this  paper  covers  recent 
AF-SD  Parts  Program  Experience. 

Recent  Parts  Program  Experience  on  the 
DSCS  ni  Program 

As  part  of  the  DSCS  III  Phase  2  program, 
which  is  a  fixed  price  contract  with  AF-SD, 
General  Electric  planned,  implemented,  and 
managed  a  Program  Integrated  Parts  System, 
called  PIPS.  The  PIPS,  work  flow  is  shown  in 
Figure  2.  A  detailed  snyposis  of  the  scope 
extent  of  PIPS  is  shown  in  Table  3.  From  the 
standpoint  of  statistics,  almost  500  types  of 
different  parts  were  procured  from  some  90 
different  parts  manufacturers;  almost  5,000 
line  items  were  ordered;  almost  400, 000  dis¬ 
crete  parts  were  screened,  burned-in,  and 
acceptance  tested;  over  a  quarter  million 
active  devices  received  on-site,  precap  visual 
inspections  as  part  of  the  Quality  Monitoring 
Program  as  shown  in  Table  4. 

Over  500  lots  of  active  devices  received  lot  sam¬ 
ple  radiation  tests  (flash  X-ray,  neutron  and 
gamma  radiation  e^qtosures);  almost  30,000 
Junction  Isolated  Integrated  Circuits  were 
given  latch-up  tests.  More  than  50  hybrid 
circuit  types  and  10  small  integrated  compon¬ 
ent  types  were  also  subjected  to  screening  and 
bum-in,  acceptance  tests,  and  radiation  tests. 

About  95%  of  the  parts  were  supplied  by  Gen¬ 
eral  Electric  to  the  users  of  PIPS.  The  reason 
the  number  was  not  100%  was  because  the 
basic  rule  of  PIPS  was  that  a  part  would  be 
supplied  from  PIPS  if  it  met  one  of  the  follow¬ 
ing  criteria: 

•  Was  used  in  any  design  provided  by 
the  prime 

•  Was  used  in  any  design  of  at  least 
two  users,  i.e. ,  GE  and  another  sub¬ 
contractor  or  two  subcontractors 

•  Was  being  provided  at  a  subcontrac¬ 
tor's  request  because  he  did  not  have 
the  capability  to  do  the  procurement 
himself 

^  Was  being  provided  to  the  subcontrac¬ 
tor  to  protect  the  product  delivery 


schedule  either  at  his  request,  or 
contractually  agreed  during  negotia¬ 
tions  with  the  subcontractor 

0  Was  a  high  technology  item  which 

General  Electric  produced,  such  as  a 
hybrid  circuit  or  needed  GE  expertise 
to  define,  procure  and  acceptance 
test  such  as  a  GaAs  Field  Effect 
Transistor 

Several  typical  hybrid  circuits  produced  at  GE 
are  shown  in  Figure  3.  From  a  review  of  the 
above  factors,  resulted  a  small  residue  of 
parts  which  fell  into  the  following  categories; 

•  A  given  subcontractor(s)  was  the  only 
user  of  a  certain  part  type.  An  ex¬ 
ample  would  be  a  very  high  ohmic 
resistor. 

•  A  given  subcontractor(s)  was  re¬ 
sponsible  for  building  that  device  as 
part  of  his  particular  product  design; 
examples  include  magnetics,  trans¬ 
formers,  chokes,  etc. 

•  A  given  subcontractor  required  a 
part  already  on  PIPS  but  with  a 
special  selection  factor  above  and 
beyond  that  required  by  the  other 
users.  There  were  only  three  such 
cases,  each  of  which  were  handled 
separately. 

The  above  items  were  handled  by  the  respon¬ 
sible  subcontractor  subject  to  the  parts  pro¬ 
gram  requirements  described  in  applicable 
pass  down  requirements  documents  traceable 
to  SAMSO  Standard  73-2B  and  the  approval  of 
the  PMPCB-Parts,  Materials,  and  Processes 
Control  Board.  The  PMPCB  composed  of  rep¬ 
resentative  from  AF-SD,  RADC,  DESC,  Aero¬ 
space,  AFPRO,  and  GE  as  well  as  subcon¬ 
tractor  representatives  when  required  meet 
over  20  times  during  the  DSCS  HI  Phase  2  pro¬ 
gram  to  discuss  and  act  upon  parts  activities 
such  as:  specification  reviews  and  approvals, 
DPA  results,  screening  results,  LAT  results, 
problems  and  corrective  action,  and  PIPS 
status. 

The  PIPS  effort  did  not  have  a  precontract 


Parts  Stocking  Program  (PSP)  phase  as  the 
design  was  not  fully  defined  at  that  point  and 
the  choice  of  a  contractor  was  not  established 
until  completion  of  a  competitive  Program 
Definition  Phase,  and  the  amount  of  funds  for 
such  an  effort  were  not  available  in  the  offers 
of  the  contractor.  As  a  result,  the  PIPS  effort 
spanned  a  significant  part  of  the  program  span. 
A  properly  managed  Parts  Stocking  Program 
could  have  reduced  this  time  cycle  on  the 
Phase  2  contract  and  would  significantly  reduce 
the  delivery  cycle  of  production  units  in  the 
Phase  3  effort. 

Pro's  and  Cons  of  a  Parts  Stocking  Program 

As  previously  discussed  in  this  paper,  a  Parts 
Stocking  Program  requires  the  establishment 
and  disciplined  use  of  an  Approved  Parts  List 
ly  Design  Engineers.  The  Approved  Parts 
List  defines  what  items  are  to  be  stocked  and 
future  anticipated  needs  establish  how  maiqr  of 
each  of  the  items  on  the  Approved  Parts  List 
will  be  maintained  in  stock.  This  then  defines 
the  materials  dollars  required  and  the  corre¬ 
sponding  labor  dollars  to  be  spent  to  establish 
the  initial  inventory  of  parts.  Real-time  rec¬ 
ords  of  parts  issued  from  inventory  and  the 
balances  remaining  by  part  type  will  trigger 
the  reordering  process  when  minimum  reserve 
quantities  are  obtained  in  stock. 

While  the  cost  savings  to  future  new  programs 
or  follow-on  production  contracts  can  be  eas¬ 
ily  shown  to  result  from  appropriate  Parts 
Stocking  Programs  which  is  a  "Pro"  for  such 
an  effort,  the  "Con"  is:  "Where  does  the  front 
end  money  come  from  to  finance  the  effort 
that  will  ultimately  pay  for  Itself  many  fold?" 
The  Catch  22  comes  into  play  as  noted  earlier, 
unless  the  end  customer(s)  helps  the  space 
contractors  with  financial  aid. 

The  financial  benefits  to  the  government,  which 
accounts  for  both  NASA  and  AF-SD  Space  Pro¬ 
jects,  of  providing  funding  to  space  contractors 
from  maintaining  an  inventory  of  critical  parts 
for  many  programs  beyond  those  funded,  can 
be  substantial.  The  parts  to  stock  must  be 
those  ^llcable  to  such  space  projects  and  as 
such  factor  in  those  parts  on  PPL- 14  for 
NASA,  AF-SD's  preferred  parts  list  of  MIL- 
STD-1546  and  MIL-STD  1547  and 
parts  on  the  lists  from  other  users  like 


JPL  and  those  parts  on  the  DESC-AQL's  as 
apprt^riate.  With  the  future  widespread  im¬ 
plementation  of  the  Class  "S"  lines  per  MIL- 
STD-1546  and  MIL-STD-1547  and  the  AF-SD 
commander's  Policy  on  Parts  of  1978,  it  be¬ 
comes  more  imperative  that  space  contractors 
prepare  and  propose  their  listings  of  approved 
Parts  Lists  for  long-life  Space  Programs  to 
their  customers  for  approval  and  use  these  as 
the  basis  for  planning  a  Parts  Stocking  Pro¬ 
gram.  Approval  of  these  listings  by  various 
customer  houses,  such  as  AF-SD  and  NASA 
centers,  and  the  issuance  of  funding  from 
these  houses  to  space  contractors  to  pay  a  fair 
share  portion  of  the  effort  will  go  a  long  way 
to  making  Parts  Stocking  Programs  an  effec¬ 
tive  reality. 

The  end  result  of  such  actions  can  only  save 
precious  dollars  and  improve  and  protect  de¬ 
livery  schedule  of  end  products.  In  a  sense, 
the  thought  is  similar  to  the  one  given  by  the 
gasoline  station  mechanic  who,  in  that  famous 
TV  commercial  says,  "You  can  pay  me  now  or 
pay  me  later. "  The  point  to  be  made  is  to  do 
it  now. 

Conclusions 

Based  on  General  Electric's  e3q>erience  with 
coordinated  and  Integrated  parts  programs  and 
the  state  of  the  marketplace  on  parts.  Parts 
Stocking  Programs  offer  definate  cost  advant¬ 
ages  above  and  beyond  those  already  being 
realized  from  the  coordinated  and  integrated 
parts  program  efforts. 

Recommendations 

The  Customer  Community  for  Space  Parts  (AF- 
SD  and  NASA  Centers)  together  with  the  Space 
Contractors  and  Parts  Producers  should  form 
a  small  ad  hoc  Studty  Group  under  government 
chairmanship  to  establish  the  plans  and  mech¬ 
anics  and  recommendations  for  effecting  Gov¬ 
ernment  Funding  assistance  programs  for 
S^ace  Parts  Stocking  Programs  by  Space  Con¬ 
tractors.  This  Mission  Assurance  Conference 
offers  a  good  place  and  time  to  kickoff  such  an 
effort.  We  at  General  Electric  Space  Division 
are  available,  and  I  am  sure  other  contractors 
are  also  available,  to  discuss  this  program 
further. 
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Figure  1.  GE  Experience  on  International  Programs 
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Figure  2.  PIPS  Work  Flow 


Table  1.  GE  Parts  Experience 
on  U.S.  Space  Programs 


How  much  can  be  save  on  an  Integrated  Parts  Program?  -  A  Case  History 
Nlmbus-G  Unified  Parts  Procurement  Program  -  Material  Cost  Analysis 


Line  Items 

Parts  Quantity 

Materials-K  $  I 

UPPP-Mgr 

6  Users 

UPPP-Mgr 

6  Users 

UPPP-Mgr 

6  Users 

Microcircuits 

105 

267 

14,344 

9,356 

645.3 

Transistors 

34 

98 

4,943 

3,766 

56.9 

Diodes 

69 

134 

11,480 

9,123 

81.7 

93.2 

Relays 

9 

21 

599 

436 

55.4 

61.2 

Capacitors 

214 

367 

13,420 

11,622 

66.5 

93.5 

Resistors 

964 

1,612 

26,212 

23,210 

52.2 

76.7 

Connectors 

58 

96 

1,220 

993 

21.8 

22.2 

Other  Misc. 

27 

47 

2,771 

66.6 

71.1 

TOTALS 

1,480 

2,642 

74, 989 

60,571 

1,046.4 

1,240.2 

Materials.  Average  Unit  Cost 

$13.95 

$20.48 

I  (32%  Average  Unit  Cost  Savings) 
f  More  Parts  for  Fewer  Dollars  and  Schedule  Protection  | 


Table  2A.  Parts  Program 
Savings  Materials 


NimbuB-C  Unified  Parts  Procurement  Program 
-  Total  Cost  Material 


6  Different 
Users  Each 
Manage  Their 
Parts  Program 

Unified  Parts 
Procurement 
Program 
Management 

Unified  Parts 
Procurement 
Program 
Management 

Cost 

Cost 

Cost  Savings 

• 

Specification 

Preparation 

$  168. 3  K 

$  100.7  K 

$  67 

6  K 

• 

Procurement/ 

Expendltlng 

266.3 

114.4 

1.61.9 

• 

Prccap 

Inspections 

34.6 

14.4 

20.2 

• 

Acceptance 
InspecUon 
and  Test 

218.2 

209.7 

8 

5 

• 

Materials 

(Table  2A) 

1,240.2 

1,046.4 

193.8 

• 

UPPP 

Integration 

1,110.0 

(116.0) 

Totals 

$1,927.60 

$1,601.60 

326.  OOK 

Total  Savings 

$  326.  OOK 

Savings  In  Labor  and  Materials  Dollars  are  Significant 

Table  2B.  Parts  Program  Savings, 
Total  Costs 


•  User  Identifies  parts  needs  to  prime  con¬ 
tractor 

•  Prime  contractor  consolidates,  stan¬ 
dardizes  and  Integrates  requirements 
with  users 

•  Prime  contractor  specifies,  orders,  ex¬ 
pedites  and  delivers  all  common  usage 
parts  for  13  subcontractors  making  21  dif¬ 
ferent  products.  In  addition  to  parts  need¬ 
ed  by  prime  for  his  designs 

SUBCONTRACTOR 

DESIGNS  SUBCONTRACTORS 


LINE  ITEMS  INVENTORY  TYPE  ON-BOARD 

•  Significant  cost  reductions 

Unit  costs  less 
Fewer  lot  charges 

Elimination  of  duplicate  labor  efforts 
Reduction  of  travel  and  living  costs 

•  Uniform  quality  program 

Common  process  reviews  and  surveys 
Fewer  sources  of  su[q)ly 
Greater  traceability  of  data 

•  Schedule  maintenance 

Timely  ordering 

Program  management  visibility 

Early  problem  resolution 


Table  3.  PIPS  Scope 


Number 

No.  of  Parts 

GE  Precap 

Inspection  Results  (CSb 

Of 

Accepted  by 

No.  of  Parts 

No.  of  Parts 

Percent  of 

Part  Type 

Lots 

Manufacturers 

Accepted 

Rejected 

Rejections 

Diodes 

145 

94,183 

92, 765 

1,418 

1.50 

ICS 

681 

133,639 

130,238 

3,401 

2.54 

Transistors 

105 

26,750 

26,299 

451 

1.68 

Total  Actives 

931 

254,  572 

249, 302 

5,270 

2.07 

Capacitors 

19 

2,223 

2,190 

Hi 

1.48 

Crystals 

20 

250 

246 

HB 

1.60 

Filters 

8 

2,479 

2,463 

16 

0.64 

Relays 

53 

5,742 

5,558 

184 

3.20 

Total  Passives 

100 

10,694 

10,457 

237 

2.21 

Other 

51 

2,121 

2,034 

87 

OIS 

Grand  Totals 

1,082 

267,387 

261,793 

5,594 

2.09 

Table  4.  Precap  Visual  Inspection 
Results 
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Introduction  -  This  presentation  will 
deal  briefly  with  considerations  to  be 
addressed  when  hybrid  parts  are  to  be 
used  in  electronic  systems/subsystems. 

The  issues  to  be  addressed  are  as  fol¬ 
lows: 

®  When  are  system/subsystem  management 
techniques  appropriate  for  a  hybrid 
procurement? 

®  What  types  of  trade-offs  are  to  be 
evaluated  In  the  decision  to  use 
hybrid  parts? 

^  What  types  of  management  controls 
are  needed  for  hybrid  procurements? 

®  How  can  test  capabilities  be  fully 
utilized  In  hybrid  procurements? 

The  actions  suggested  herein  have  In  most 
cases  been  used  In  actual  procurement 
programs,  or  are  those  which  experience 
has  shown  would  be  desirable  for  Imple¬ 
mentation  on  new  programs.  They  are  not 
intended  to  delineate  specific  technical 
requirements,  but  to  act  as  signposts 
along  the  development/production  path. 

The  usage  of  hybrid  parts,  usually  but 
not  always  In  the  form  of  microcircuits, 
has  become  Increasingly  popular  In 
system  design.  Consideration  of  the 
density  of  circuit  elements  that  can  be 
accommodated  In  a  typical  hybrid  almost 
begs  the  question  of  whether  or  not  a 
hybrid  should  be  considered  as  a  sub¬ 
system.  Circuit  partitioning  of  a  system 
Into  a  group  of  hybrids  almost  always 
results  In  the  definition  of  functional 
blocks  (or  subsystems)  of  a  system,  and 
these  hybrids  historically  requi re  manage¬ 
ment  as  subsystems. 

The  cost  trade-offs  that  are  performed 
In  the  determination  to  use  hybrids  are 
of  the  usual  kinds:  Wei ght/vol ume ,  time, 
and  funds.  The  wel ght/vol ume  tradeoff 


can  be  approximated  from  a  conservative 
baseline  of  approximately  47  circuit 
elements  per  square  Inch  of  hybrid  sub¬ 
strate  area.  System  penalties  associated 
with  wel ght/vol ume  can  usually  be  readily 
assessed.  The  trade-offs  In  time  center 
around  the  efforts  for  system  parti¬ 
tioning,  hybridization  feasibility, 
prototype  development/test,  element  pro¬ 
curement/test,  and  hybrid  production/ 
test.  The  funding  tradeoffs  are  subtle, 
and  related  to  the  system  contractor's 
capability  in  the  areas  of  design, 
production,  and  test  as  related  to  hy¬ 
brids;  the  required  quality  level  of 
parts  for  the  system  and  the  prevailing 
parts  market  situation' are  also  factors. 
Realistic  consideration  of  these  trade¬ 
offs  Is  not  at  all  straightforward,  and 
the  deciding  factor  is  usually  the  most 
Inflexible  requirement  of  the  contract 
specification  for  the  system.  Discussion 
of  a  typical  tradeoff  Issue  Is  presented 
In  Figure  1. 

The  management  controls  required  for 
development  and  procurement  of  hybrid 
parts  are,  fortunately,  not  unique. 
Particular  emphasis  should  be  focused  in 
three  areas:  hybrid  specification, 
supplier  evaluation,  and  project  moni¬ 
toring.  A  brief  review  of  each  of  these 
areas  of  management  control  will  high¬ 
light  some  concerns. 

Specification  -  The  first  ^act  to  be 
realized  Is  that  there  Is  no  existing, 
coordinated  specification  that  can 
be  used  as-ls  for  hybrid  procurement. 
The  system  must  be  partitioned  Into 
functional  hybrids  and  approved 
through  the  normal  rigor  of  design 
review.  Performance  tests  must  be 
defined,  and  test  software  developed. 
Although  It  Is  not  usually  desirable 
to  define  a  particular  hybrid  fabri¬ 
cation  process  In  the  specification 
(because  It  may  lock  out  many 
potentially  good  hybrid  suppliers), 
certain  basic  process  constraints 
that  have  a  bearing  on  process  yields 
should  be  defined;  for  example.  Items 
such  as  allowable  rework,  delldding, 
coatings,  organic  materials,  metal 
joining  systems,  and  sealing  tech¬ 
niques  should  be  addressed.  Screening 
tests,  both  In-process  and  as  final 
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acceptance,  may  need  to  be  uniquely 
defined  for  hybrids;  examples  might 
be  die-bonding  and  wire-bonding  In¬ 
line  testing,  water  vapor  content  In 
sealed  units,  electrolytic  stress 
tests,  particle  noise  tests,  pre-seal 
burn-in  tests,  thermal  mapping  tests, 
and  operational  temperature  cycling 
burn-in.  The  qualification  of  the 
hybrids  occurs  In  essentially  two 
phases;  the  verification  of  the  cir¬ 
cuit  design,  and  the  assessment  of 
the  hybrid  fabrication  process.  The 
circuit  verification  should  largely 
be  accomplished  before  the  major 
hybrid  production  is  committed,  and 
is  generally  done  via  breadboard  and 
prototype  hybrid  tests.  Hybrid  fab¬ 
rication  process  assessment  and 
certification  can  be  accomplished  in 
piecemeal  fashion  to  a  large  extent, 
in  that  each  production  process  can 
be  evaluated  Independently,  and  also 
as  a  series  of  In-line  production 
tests.  As  a  minimum,  the  objective 
evidence  required  by  the  Product 
Assurance  Program  of  Appendix  A  of 
M11-M-38510  should  be  Imposed.  The 
specification  should  require  a  pro¬ 
cess  baseline  definition,  and 
approval  by  the  buyer  of  any  proposed 
process  changes  (this  also  implies  an 
objective  evaluation  and  approval  by 
the  buyer  of  any  reasonable  proposed 
change,  when  supported  by  sufficient 
test  data).  The  element/circuit  de¬ 
rating  criteria  should  be  Included  In 
the  specification,  and  a  detailed 
thermal  analysis  should  be  required. 
Finally,  the  degree  of  process, 
material,  and  element  traceability 
for  the  hybrid  should  be  defined  by 
the  specification. 

Supplier  Evaluation  -  Whether  the 
hybrid  supplier  Is  an  outside  vendor 
(usually  a  specialty  house)  or  an  In- 
house  facility  (generally  dedicated), 
an  objective  evaluation  of  supplier 
capabilities  must  be  performed.  The 
Initial  step  Is  to  evaluate  the  capa¬ 
bility  to  produce  the  volume  of 
hybrids  in  the  allotted  schedule  time. 
This  can  be  approximated  by  detennlna- 
tlon  of  total  circuit  element  popula¬ 
tion,  Including  wire  bonds,  and  com¬ 
parison  of  these  populations  to  the 


demonstrated  thru-put  rates  of  the 
supplier  (inspection  thru-put  rates 
are  also  essential  in  this  evalua¬ 
tion).  The  testing  rate  for  both 
electrical  and  environmental  tests, 
and  the  anticipated  yields  must  also 
be  factored  into  the  overall  assess¬ 
ment.  It  is  a  mistake  to  believe 
that  capacity  can  be  quickly  and 
easily  expanded,  even  for  a  mature 
and  proven  process.  Next,  the 
technical  capability  of  the  supplier 
must  be  evaluated,  both  in  terms  of 
circuit  design  and  process  compati¬ 
bility.  These  two  factors  are  inter¬ 
related,  in  that  the  supplier  may  be 
able  to  compensate  for  his  process 
characteristics  by  circuit  design 
alterations;  he  should  be  able  to 
demonstrate  by  test  data  the  charac¬ 
teristics  and  tradeoffs  associated 
with  his  process  and  materials. 

Offers  to  implement  the  buyer's  de¬ 
sign  with  non-mature  processes/ 
materials  should  be  approached  with 
extreme  caution.  In  line  with  this 
consideration,  the  supplier  should 
be  able  to  demonstrate  by  production 
and  test  data  the  maturity  of  his 
standard  process,  and  its  expected 
yields.  Incredible  as  it  may  seem, 
many  suppliers  do  not  understand 
their  own  processes  well  enough  to 
demonstrate  such  proof.  If  the  sup¬ 
plier  is  being  considered  as  a 
second  source  for  a  qualified  hybrid, 
careful  consideration  must  be  given 
to  the  compatibility  of  his  process 
with  respect  to  the  hybrid  design. 
Such  innocuous  differences  as  wire 
bonding  techniques,  process  tempera¬ 
tures,  package  seal  techniques,  and 
package  plating  may  have  far  reaching 
effects  on  a  particular  hybrid  cir¬ 
cuit's  performance  and  reliability. 

An  example  of  problems  actually 
associated  with  a  multiple  source 
procurement  is  shown  in  Figure  2. 

Project  Monito''ng  -  If  the  procure¬ 
ment  of  hybricj  is  not  second-sourced, 
adequate  project  monitoring  is  re¬ 
quired  to  anticipate  and  circumvent 
the  problems  that  are  typical  with 
most  hybrid  projects.  Schedule  prob¬ 
lems  are  almost  endemic,  and  are  re¬ 
lated  to  such  things  as  extended 
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delivery  on  semiconductor  elements, 
element  lot  evaluation/acceptance 
problems,  rework,  "black-art"  fabri¬ 
cation  problems,  and  the  "ever 
popular"  equipment  downtime.  If  any 
management  control  or  Influence  1s 
to  be  realistically  exercised  on  a 
hybrid  production  contract.  It  Is 
essential  that  a  real-time  reporting 
system  of  work-in-progress  and  yield 
(or  thru-put)  be  established  with  the 
hybrid  supplier.  Experience  has 
shown  that  an  effective  method  Is  to 
require  reporting  of  all  Inspection 
actions  on  the  hybrid  production  line 
on  a  weekly  basis  (see  Figure  3). 
Additionally,  the  requirement  for 
analysis  of  failures  and  Institution 
of  corrective  action  should  be  re¬ 
quired  at  the  earliest  possible  point 
in  the  hybrid  manufacturing  cycle. 

The  long  lead  times  for  hybrids, 
coupled  with  the  system  assembly/test 
times,  make  it  unrealistic  to  assume 
that  any  meaningful  in-line  corrective 
action  can  be  made  In  hybrid  pro¬ 
cesses  if  the  user  waits  for  box- 
acceptance  tests  and/or  field  failures 
to  trigger  the  change  actions.  It 
has  been  found  cost-effective  to  re¬ 
place  the  requirement  for  destructive 
physical  analysis  of  hybrids  with 
customer/government  source  inspection; 
this  action  also  allows  for  a  more 
effective  chance  to  institute  in-line 
corrective  actions  on  the  questionable 
hardware/process.  The  final  monitor¬ 
ing  point  would  obviously  be  cost/ 
hours  on  a  contract  other  than  fixed 
price. 


A  final  comment  on  possible  Improvements 
In  test  capability  appears  to  be  In  order. 
The  thru-put  of  hybrid  production  can  of 
course  be  greatly  enhanced  by  the  stan¬ 
dardization  of  the  functions  to  be 
hybridized,  and  by  allocating  as  much 
testing  as  possible  to  not  only  stan¬ 
dardized  ATE,  but  to  test  programs 
associated  with  laser  resistor  trimmers. 
The  thru-put  rate  of  these  types  of  equip¬ 
ment  Is  quite  high,  and  should  be  used  to 
the  maximum  extent  on  any  hybrid  procure¬ 
ment.  In  addition,  the  use  of  functional 
trimming  can  in  many  cases  alleviate  long 
lead  time  problems  on  procurement  of  pre¬ 
cision  resistor  arrays  and  exotic 
functional  semiconductor  devices. 


-  . 
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In  summary.  It  Is  recommended  that  all 
except  the  very  simple,  off-the-shelf 
hybrid  procurements  should  be  the  subject 
of  stringent  management  attention/ tech¬ 
niques  that  are  more  commonly  associated 
with  subsystems. 


EXAfm  OF  TRAOe-OFF  COHSIDERATrOHS 
FOR  HYBRID  USAGE 


ISSUE:  PART  REPLACEflENT  AT  MARD/BOX  LEVEL 

ITEW  DISCRETE  PART  HYBRID  PART 

REPUCEflEHT  COST  LOW  TO  NEDIWI  nEDItfl  TO  VERY  HIGH 

REHOVAL  DAHAGE  RISK  LOM  LOM  TO  HIGH 

TO  BOARD 

ANALYSIS  OF  RENOVED  NO  REWORK  AT  PART  TROUBLESHOOT  TO  ELENENT 

PART  LEVELi  PERFORM  (PART)i  DECIDE  WHETHER 

ANALYSIS.  TO  REWORK  HYBRID.  DR 

SACRIFICE  HYBRID  TO 
ANALYZE  ELEMENT. 


EXAMPLE  OF  MULTfPLE-SOURCE  PROBLiMS 


•  ORIGINAL  SOURCE 

•  LOH  TEMPERATURE  GOLD  ULTRASONIC  fiAU  BONDING 

•  THICK  FILM  GOLD  SUBSTRATE  METAUIZATION 

•  EPOXY  SUBSTRATE  ATTACHMENT 

•  VACUUM  BAKEOUT  THRU  SEAL  WITHOUT  EXPOSURE  TO  ATMOSPHERE 

•  DIE  ACCEPTANCE  VIA  CANNED,  WIRE  BONDED  SAMPLES 

•  SECOND  SOURCE 

•  THIN  PIIM  TITANIUM-PALLADIUM-GOU)  SUBSTRATE  METALLIZATION 

•  EPOXY  SUBSTRATE  ATT\CH  MANDATEDi  NORMALLY  USED  SOLDER 

•  VACUUM  BAKEOUT  FOLLOWED  BY  ATMOSPHERIC  EXPOSURE  PRIOR  TO  SEAL 

PROBLEM  -  CONTAMINATED,  MOISTURE-LADEN  HYBRIDS  EXPERIENCED  SHORTS  DUE 
TO  GOLD  DENDRITES  AT  BOX-LEVEL  THERMAL  TESTS 

•  THIRD  SOUM 

•  ALUMINUM  ULTRASONIC  HEDGE  BONDING 

'  lOOZ  DIE  PROBE  TESTING  FOR  ACCEPTANCE 
PROBLEM  -  CRATERED  TRANSISTOR  DICE  UNDER  WIRE  BONDS  EXPERIENCED 


rrpicAi  wwg  m  htbup  usswy 


*  MEICMT  OF  (IKKS  III  PtODUCT  FIOM  LHC  IS  WtCATIVE  OF  IHRU-PUT 

*  TYPIGM.  AWOTITIORS  F(»  MN  FLOCIT  MMJ’  UDICATE: 

•  WUT5  TO  SI^TIOW  PCR 

•  WITS  (CJtCTU  FtR  WW 

•  STATICi  YIELD  (1)  PER  ICEK 
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MANAGEMENT  APPROACH  TO  HANDLING 
CUSTOM  LSI 

w  Richard  Scott 

Supervis; '  ,  Parts  Engineering  Group 
Jet  Propulsion  Laboratory 

As  defined  by  Meriam  Webster,  the  term, 
'approach'  implies  the  taking  of  highly 
Individual,  preliminary  steps  toward  a 
particular  purpose.  The  approach  we  are 
taking  in  an  on-going  custom  LSI  program 
at  JPL,  while  it  may  be  a  specific  set 
of  steps  toward  a  particular  purpose, 
may  still  be  applicable  to  your  organiza¬ 
tions  and  for  that  reason  1  would  like 
to  share  with  you  some  of  the  experiences 
from  which  we  have  learned  and  are  still 
learning.  Following  this  "Share  and 
Tell"  I  would  like  to  discuss  some  new 
approaches  we  are  studying  at  JPL  to 
make  future  custom  LSI  programs  more 
successful. 

BACKGROUND 

In  order  to  understand  our  approaches 
we  need  to  examine  the  kinds  of  programs 
which  we  are  supporting  at  JPL.  JPL  has 
been  given  by  NASA  the  prime  responsi¬ 
bility  for  NASA's  unmanned,  planetary 
spacecraft  programs.  Since  the  1960's, 
JPL  has  been  responsible  for  the  Ranger, 
Surveyor,  Mariner  and  Viking  programs, 
which  sent  spacecraft  to,  past,  or  orbit¬ 
ing  about  the  Moon  and  Mars  as  well  as 
Venus  and  Mercury.  More  recently  we 
conducted  the  dual-spacecraft  Voyager 
program  whose  spacecrafts  encountered 
Jupiter  last  year,  will  reach  Saturn  in 
about  seven  months  and  fly  by  Uranus  in 
1986.  The  Voyager  was  designed  to  meet 
requirements  of  several  years  of  opera¬ 
tional  life,  long-range  communications 
(AGO  -  1700  million  miles),  precision 
navigation,  solar-independent  power, 
resistance  to  the  radiation  fields  of 
Jupiter,  and  to  provide  in-flight  re¬ 
programming  capability  to  support  the 
science  investigations.  JPL  is  presently 
developing  the  Galileo  spacecraft  which 
will  be  launched  in  198A.  This  mission 
begins  with  a  1000-day  journey  to  Jupiter 
followed  by  a  20-month  orbital  tour  of 
Jupiter  and  its  moons.  If  the  space¬ 
craft  survives  for  this  period,  as  is 


expected,  an  extended  mission  could  then 
continue  to  explore  the  environment  of 
the  planet.  Recognizing  the  long-life, 
non-mainta inability  requirements  of  JPL's 
spacecraft  programs,  should  put  our  cus¬ 
tom  LSI  activities  into  proper  prospec¬ 
tive. 


SELECTING  LSI 

There  are  a  number  of  choices  available 
to  designers  who  wish  to  use  LSI,  in¬ 
cluding  1)  handcrafted  off-the-shelf 
microprocessors  (1802,  Z80,  6800,  Z8000 
and  680000)  together  with  their  support 
chips;  2)  standard  cell  custom  LSI; 

3)  custom  gate  arrays;  and  4)  structured 
design  custom  LSI.  Each  has  its  place, 
its  advantages  and  disadvantages. 

On  Project  Galileo,  JPL  selected  the 
first  approach,  the  off-the-shelf  micro¬ 
processor.  However,  we  did  not  stop 
there.  Our  spacecraft  design  demanded  a 
low  power,  radiation -hard  microprocessor. 
These  conflicting  requirements  led  to 
the  selection  of  the  low-power  CMOS 
technology  which,  in  turn,  pointed  dir¬ 
ectly  to  the  1802  microprocessor  family. 
While  the  1802  was  not  radiation  hard. 
Sand la  Laboratories  had  already  begun  a 
program  of  hardening  this  RCA  device 
family.  How  fortunate  for  JPL,  a  rad- 
hard,  low-power  microprocessor.  And  it 
looked  like  we  were  home  free.  But  were 
we  home  and  was  it  free? 

We  think  our  choice  of  the  Sandia-modi- 
fied,  RCA-designed  1802  LSI  family  was  a 
wise  selection  and  eventually  will  be 
proven  sound.  But  our  pathway  has  been 
full  of  obstacles  and  knowing  of  these 
obstacles  may  make  your  stroll  down  the 
custom  LSI  road  a  little  less  hazardous. 

WARNINGS  TO  THE  WISE 

Custom  LSI  procurements  are  often  by 
nature  sole  source.  With  that  comes  the 
many  problems  which  make  sole  source 
risky  for  any  procurement  —  the  runaway 
costs  and  uncontrollable  schedules  as 
the  manufacturer  learns  how  the  customer '  s 
requirements  push  the  state  of  the  art; 
lack  of  an  adequate  back-up  position; 
changing  quantity  requirements;  and  many 
more.  A  truly  custom  device  usually 
implies  that  there  has  been  no  previous 


physical  evaluation  or  electrical  charac¬ 
terization,  much  less  qualification.  So 
the  user  doesn't  know  what  the  device 
specification  really  is  —  only  what  it 
is  targeted  to  be.  With  this  comes  the 
added  need  for  increased  technical  staf¬ 
fing,  both  be  the  device  manufacturer  as 
well  as  the  user.  Staffing  to  be  applied 
to  the  task  of  studying  the  device  in 
detail  sufficient  to  recognize  its  short¬ 
comings,  understand  their  impact  on  the 
hardware  design,  and  assess  the  potential 
risks. 

Also,  because  this  is  a  custom  device, 
the  manufacturer  has  not  fabricated  it  in 
sufficient  quantity  to  verify  his  design 
nor  work  out  his  fabrication  start-up 
problems.  Additionally,  the  design 
engineers  must  be  prepared  to  accept  and 
deal  with  the  inevitably  reduced  perfor¬ 
mance  and  reliability  characteristics. 
They  must  design  tolerant  and  perhaps 
redundent  systems  which  are  capable  of 
operating  at  reduced  voltage  levels  and 
speeds. 

In  the  present  semiconductor  market,  it 
is  difficult,  if  not  impossible,  to  per¬ 
suade  the  semiconductor  manufacturer  to 
make  the  necessary  heavy  committment  to 
an  area  of  such  marginal  profit.  And  to 
make  this  committment  for  him,  as  in  the 
case  of  Sandia  in  the  Sandia/RCA  radiation 
hardening  program,  you  are  still  left 
with  the  enormous  problem  of  the  technol¬ 
ogy  transfer  from  the  research  and  devel¬ 
opment  laboratory  of  one  company,  to  the 
production  facilities  of  another. 

We  at  JPL  have  experienced  various 
degrees  of  all  of  these  above  problems. 
And,  only  working  the  "Human  Equation" 
as  stated  yesterday  by  General  Henry, 
through  the  deep  personal  committments  of 
many  JPL,  RCA  and  Sandia  personnel  will 
we  ultimately  succeed  in  this  custom  LSI 
program,  but  not  without  having  paid  a 
high  price,  a  price  measured  not  only  in 
dollars  but  also  in  lost  sleep,  weight 
and  a  lot  of  other  scarce  resources. 

THERE  IS  A  PLACE  FOR  CUSTOM  LSI 

We  are  finding  that  the  use  of  off-the- 
shelf  LSI,  such  as  the  1802  microproces¬ 
sor,  requires  inordinate  amounts  of  small 
scale  and  medium-scale  Integrated  circuits 


for  various  interface  functions  which 
could  be  easily  implemented  in  one  to 
two  custom  chips.  In  the  Galileo  design, 
for  instance,  one  such  interface  function 
requires  on  the  order  of  forty  off-the- 
shelf  chips. 

In  the  far  term,  we  see  in  our  applica¬ 
tions,  the  need  for  processing  at 
throughput  rates  orders  of  magnitude 
greater  than  those  presently  available 
with  conventional  microprocessors.  For 
example,  construction  of  geometrical 
images  from  snythetlc  aperture  radar 
data  in  real  time  requires  over  ten 
billion  complex  operations  per  second. 

A  CUSTOM  LSI  PROGRAM 

Driven  by  these  near  and  far-term  re¬ 
quirements,  we  are  developing  at  JPL 
what  we  call  our  "Custom-LSI  Headstart 
Program"  (CHP).  Under  this  program  we 
are  educating  design  engineers  in  the 
fundamentals  of  semiconductor  design; 
developing  the  computer  design  aids  nec¬ 
essary  to  design  custom  LSI  devices; 
establishing  wafer  fabrication  sources; 
and  developing  a  product  assurance 
approach  which  will  assure  that  the 
resultant  custom  LSI  is  free  of  workman¬ 
ship  defects  and  is  reliable  enough  for 
spacecraft  applications. 

Under  this  program  designers  are  trained 
by  their  participation  in  a  design 
course  based  on  a  course  currently 
offered  at  Caltech  by  Professor  Carver 
Mead.  The  course  teaches  the  fundamen¬ 
tals  of  structured  design  at  the  pattern¬ 
ing  level  and  encompasses  both  theory 
and  "hands-on"  experience.  Each  student 
has  the  opportunity  to  design  his  own 
LSI  circuit,  have  it  fabricated,  and 
then  test  it . 

The  second  goal  of  the  headstart  program 
is  to  develop  the  tools  and  the  capabil¬ 
ity  for  computer  aided  design  of  custom 
LSI.  This  requires  developing  some  new 
hardware  and  software  as  well  as  inter¬ 
connecting  the  CAD  facilities  at  JPT., 
NASA,  the  Army,  Sandia  I.aborator  ies ,  and 
others  in  order  to  facilitate  access  to 
these  newly  developed  design  aids. 

The  third  goal,  that  of  establishing 
wafer  fabrication  sources  and  controls. 


CONCLUSION 


calls  for  developing  an  approach  to  fab¬ 
rication  which  separates  the  circuit 
design  from  the  fabrication  process. 

The  designer  bears  the  full  responsibility 
for  the  design  of  a  functional  circuit 
which  will  meet  all  his  system  require¬ 
ments.  The  fabrication  facility,  in 
turn,  is  charged  with  the  responsibility 
of  fabricating  the  circuit  using  fully 
controlled  process  steps.  JPL  is  pres¬ 
ently  developing  test  chips  which, 
through  appropriate  electrical  stresses 
and  measurements,  may  be  the  means  for 
accepting  or  rejecting  the  wafers  from 
the  fabrication  organization. 

Lastly,  the  CHP  is  developing  a  product 
assurance  approach.  If  we  are  to  fly 
custom  parts,  we  must  learn  to  qualify 
them.  It  does  not  appear  that  this  can 
be  accomplished  merely  by  testing  the 
finished  product.  We  are  studying 
various  approaches  to  the  qualification 
process  Including:  requiring  ultra  con¬ 
servative  design  rules;  designing  in 
circuit  testability;  completing  signifi¬ 
cant  portions  of  the  device  qulalf ication 
in  conjunction  with  the  device  design 
verification;  using  test  chips  to  accom¬ 
plish  other  portions  of  the  qualification 
process;  and  finding  other  techniques  to 
replace  some  of  the  classical  qualifica¬ 
tion  procedures. 


Using  custom  LSI  devices  in  today's  en¬ 
vironment  can  and  does  carry  with  it 
many  problems  and  I  have  tried  to  enum¬ 
erate  some  of  these  from  our  own  exper¬ 
iences  at  JPL.  But  there  are  valid  and 
compelling  reasons  to  consider  the  use  of 
custom  LSI  in  new  hardware  designs, 
designs  in  which  the  available  off-the- 
shelf  devices  do  not  meet  the  system 
functional  requirements.  And  finally  to 
better  meet  the  reliability  challenges 
of  custom  LSI  devices,  JPL  is  developing 
its  Custom-LSl  Headstart  Program.  It  is 
only  through  such  a  system  approach  to 
LSI  design,  fabrication,  and  qualifica¬ 
tion  that  we  feel  custom  LSI  will  ever 
be  achievable,  available  and  acceptable 
for  high  reliability  applications. 


TEST  CHIPS  CAN  BE  HELPFUL 


Along  this  line  we  are  currently  evalua¬ 
ting  the  capabilities  of  various  test 
chip  structures  to  provide  direct 
measures  of  die-related  failure  modes 
such  as:  surface  contamination;  defects 
in  the  crystal  structure,  oxides  and 
passivation  layers;  and  doping,  mask  and 
metallization  faults.  We  are  also  eval¬ 
uating  test  chip  electrical  test  data  for 
correlation  with  packaged  device  electri¬ 
cal  test  results.  Our  goal  is  to  design 
test  chip  structures  and  define  electri¬ 
cal  measurements  on  those  structures  with 
which  we  can  predict  which  semiconductor 
wafer  runs  will  not  stand  up  to  the 
rigors  of  acceptance  testing  or  qualifi¬ 
cation  testing.  If  we  are  successful  in 
this  endeavor,  we  will  be  able  to  reject 
devices  at  the  wafer  stage  which  other¬ 
wise  would  not  be  rejected  until  after 
weeks  and  even  months  of  assembly,  pack¬ 
aging,  and  testing. 


EXPLORAT 1  ON  OF  THE  PLANETS 


THERE  IS  A  PLACE  FOR  CUS  TON  LSI 


•  Moon 

Ranger 

Surveyor 

•  Venus  -  Mercury 

Mariner 

•  Mars 

Mariner 

Viking 

•  Jupiter  -  Saturn  -  Uranus 

Voyager 

Galileo 

•  Sun 

International  Solar  Polar  Mission 

LSI  SELECTION  OPTIONS 

•  Off-the-shelf 

•  Standard  Cell  Custom 

•  Custom  Gate  Arrays 

•  Structured  Design 


INHERENT  PROBLENS  WHEN  USING  CUSTOM  LSI 


• 

Sole  source  procurement 

• 

Runnowoy  costs  and  schedule 

• 

Bock-up  device 

• 

Characterization 

• 

Ouol If Icotlon 

• 

Specification 

• 

Technlcol  Staffing 

• 

Hanufocturlng  start-uo  orablems 

• 

Reduced  performance 

• 

Technology  transfer 

•  To  meet  basic  requirements 

•  Interface  functions 

•  Throughput  rates  beyond  capability 

of  conventional  devices 

CUSTOM  -  LSI  HEADSTART  PROGRAM  (CHP) 

•  Educating  Custom  LSI  designers 

•  Developing  CAD 

•  Establishing  wafer  fabrication  sources 

•  Product  assurance  approach 

CUSTOM  LSI  QUALIFICATION  PROCESS 

•  Ultro  conservative  design  rules 

•  Designed  In  testoblllty 

•  Device  design  verlf Icotlon 

•  Test  chips 

TEST  CHIPS 

•  Surfoce  contomlnotlon 

•  Defects  (crystal,  doping,  and  passivation) 

•  Faults  (doping,  mask,  and  metallization) 

•  Predictor  of  electrical  and  functional  rejects 
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SUMMARY  OF  ISSUES 
AND  RECOMMENDATIONS 


9 


THE  MANAGEMENT  AND  PROCURE¬ 
MENT  OF  PIECE  PARTS 

SUMMARY  OF 

ISSUES  AND  RECOMMENDATIONS 

1.  Small  Volume  Procurement 
Issue 

Programs  and  contractors  have 
difficulty  getting  space  quality 
parts  in  small  volume  at  reason¬ 
able  cost  and  schedule. 

Recommendations 

oUSAF/SD  in  coordination  with 
NASA,  investigate  acceptable 
methods  of  up  grading  lower 
level  screened  and  Class  'B" 
qualified  parts  for  use  in  appli¬ 
cations  requiring  Class  'S''  de¬ 
vices  that  are  not  on  the  quali¬ 
fied  products  list. 

oUSAF/SD  and  NASA  examine 
techniques  for  strengthening 
standard  implementation  of  parts 
technical  requirements  across 
projects. 

2,  Class  "S"  Parts  Program 
Issue 

Class  "S"  market  volume  require¬ 
ments  are  small-manufacturers 
are  not  motivated  to  certify,  qual¬ 
ify,  and  accelerate  production  and 
reduce  lead  times. 

Recommendations 

oUSAF/SD  and  NASA  and  industry 
jointly  accelerate  efforts  to 
better  standardize  on  space 
quality  parts  requirements  across 
a  broader  set  of  users. 

SD  programs 
Com  Sat 

NASA  space  flight  programs 
Other  space  part  users 
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oUSAF/SD  and  NASA  develop 
methods  to  maximize  ease  of 
procurement  of  standard  space 
quality  parts,  e.  g.  , 

Stocking  by  gov't  or  prime 
QPL  listing 
Wafer  banking 
Long  lead  funding 

3.  Custom  LSIC 
Issue 

Custom  Large  Scale  Integrated 
Circuit  (1,SIC'  and  hybrids  can 
introduce  more  cost,  schedule, 
and  reliability  risk  than  standard 
LSIC  and  hybrids. 

Recommendations 

oUSAF/SD  and  NASA  should  assure 
that  programs  conduct  sufficient 
evaluations  and  volume /weight 
implementation  tradeoffs  to 
justify  the  use  of  custom  LSIC 
and  hybrids  when  the  design  could 
be  implemented  with  standard 
parts. 

4.  Management  of  LSIC 
Issue 

Classifying  custom  Large  Scale 
Integrated  Circuits  (LSIC'  and 
custom  hybrid  microcircuits  as 
parts  subjects  them  to  ''parts" 
controls  that  may  not  be  adequate 
or  appropriate  for  these  "system- 
like'  devices. 

Recommendations 

oUSAF/SD  and  NASA  should  assure 
that  custom  LSIC  and  custom  hy¬ 
brid  development  and  procure¬ 
ment  includes  appropriate  system 
level  controls  as  well  as  appro¬ 
priate  parts  controls  through 
management  and  technical  docu¬ 
mentation. 

Design  Review 
Qualification 
Softwa  re 
Screens 
Application 


CONCLUSIONS 


Small  Volume  Parts  Procurement 

□  Still  plague  space  programs 

o  Special  project  requirements 
are  a  major  cause  of  small 
volume  buys. 

o  Cause  cost,  no  bid,  reliability 
problems . 

oAs  Military/Aerospace  market 
percentages  decrease  small 
volume  procurement  commands 
less  attention  by  vendors. 

o  Lack  of  firm  market  forecast  for 
space  quality  parts  further  de¬ 
creases  producer  motivation. 

□  Coordinated  or  pooled  procure¬ 
ments  offer  partial  solution  to 
small  volume  procurements  and 
are  encouraged  by  USAF /SD, 
NASA  and  space  contractors. 

Coordinated  Procurement 

oSev  :al  different  approaches  used 
-varying  degrees  of  task  central¬ 
ization. 

□  Offers  cost,  schedule  and  reli¬ 
ability  advantages. 

□  Requires  more  in-depth  manage¬ 
ment. 

o  Avoid  detailed  specification  of 
coordinated  procurement  require¬ 
ments  in  contract.  Let  contract¬ 
or  explain  approach  in  proposal. 

□  Schedule  liability  is  a  major 
concern  in  implementing  co¬ 
ordinated  procurement. 

Management  of  Large  Scale  Inte¬ 
grated  Circuits  (  LSIC)  and  Hybrid 
Microelectronic  Devices 

□  The  demand  for  custom  LSIC  and 
hybrids  is  expected  to  increase  to 


meet  future  space  system  re¬ 
quirements. 

□  Custom  LSIC^hybrids  introduce 
more  cost,  schedule  and  reli¬ 
ability  risk  than  standard  LSIC^ 
hybrids. 

o  There  are  characteristics  of 
LSIC  and  hybrid  devices  that  are 
not  currently  covered  by  space 
quality  part  requirements. 

o  Hybrid  microcircuits  are  both 
parts  and  assemblies  /  sub¬ 
systems  and  must  be  managed  as 
both. 
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Future  Requirements  for  Microelectronics  in 
Commercial  C(»nmunications  Satellites* 


Irwin  A.  Feigenbaum 
COMSAT  Laboratories 
Clarksburg,  MD  20734 


Introduction 

Commercial  communications 
satellite  systems  have  developed 
into  an  effective, and  reliable 
means  of  providing  both  global  and 
domestic  communications  services. 
The  growth  and  success  of  these 
systems  have  led  to  increased  cir¬ 
cuit  capacities  and  satellite 
complexities . 

In  the  past,  commercial  com¬ 
munications  satellites  have  relied 
on  dedicated  hard -wired  logic  to 
provide  the  required  monitoring, 
switching,  and  control  functions. 
Since  future  generation  communica¬ 
tions  satellites  will  have  added 
complexity  and  require  a  signifi¬ 
cantly  larger  amount  of  such  func¬ 
tions,  the  use  of  microprocessors 
and  other  LSI  devices  is  necessary 
to  provide  added  capability,  flex¬ 
ibility,  and  reliability. 

Commercial  Communications 
Satellites 

The  world's  first  operational 
commercial  communications  satel¬ 
lite,  INTELSAT  I  (Early  Bird),  was 
launched  and  placed  in  a  geosta¬ 
tionary  orbit  35,780  km  from  the 
earth  in  1965.  It  was  capable  of 
providing  240  two-way  voice  cir¬ 
cuits  between  the  United  States 
and  Europe.  Since  that  time,  ap¬ 
proximately  110  satellites  have 
been  launched  and  placed  in  a  geo¬ 
stationary  orbit. ^  Approximately 
40  of  these  satellites  have  been 
placed  in  commercial  international 
or  domestic  communications  serv¬ 
ice,  and  the  remaining  satellites 
have  military,  experimental,  or 
meteorological  communications 
functions . 


Table  1  lists  the  commercial 
communications  satellites  cur¬ 
rently  in  service.  Of  these, 
INTELSAT  I,  II,  and  III  satellites 
have  completed  their  mission  and 
have  been  removed  from  orbit.  The 
table  also  lists  over  40  addi¬ 
tional  commercial  communications 
satellites  which  are  presently 
being  designed  and/or  constructed 
and  are  expected  to  be  launched 
during  the  next  several  years.  In 
addition,  more  than  10  new  series 
of  commercial  communications  sat¬ 
ellites  are  in  the  planning  stage 
as  of  this  date . 

The  majority  of  the  commer¬ 
cial  satellites  included  in  Ta¬ 
ble  1  provide  fixed  satellite 
services  such  as  telephone,  tele¬ 
graph,  television,  data  and  fac¬ 
simile  on  an  international,  domes¬ 
tic,  or  regional  basis.  The 
oldest  and  largest  satellite  com¬ 
munications  system  includes  the 
INTELSAT  satellites  which  provide 
services  on  an  international 
basis.  Examples  of  domestic  sys¬ 
tems  are  the  United  States ' 

Westar,  Satcom  and  Ccxnstar  satel¬ 
lites,  Canada's  ANIK,  Indonesia's 
Palapa,  and  the  Soviet  Union's 
Statsionar  satellites. 

The  other  commercial  satel¬ 
lites  shown  provide  mobile  ser¬ 
vices.  The  COMSAT  General 
Marisats  provide  combined  UHF  and 
L-band  services  to  ships  of  many 
nations.  Additional  ship  service 
will  also  be  provided  by  the  Euro¬ 
pean  Space  Agency's  Marecs  satel¬ 
lites,  and  the  Soviet  Union  is 
planning  the  use  of  Volna  satel¬ 
lites  for  mobile  services. 


*This  paper  is  based  upon  work  performed  at  COMSAT  Laboratories  under  the 
sponsorship  of  the  Communications  Satellite  Corporation. 


Table  1 . 

Commercial 

Communications 

Satellites 

Placed 

In  Service 

Satellite  Series 

First  Launch  Number 

Usage 

INTELSAT  I 

1965 

1 

International 

INTELSAT  II 

1967 

3 

International 

INTELSAT  III 

1968 

5 

International 

INTELSAT  IV 

1971 

7 

International 

INTELSAT  IV-A 

1975 

5 

International 

WESTAR 

1974 

3 

U.S.  Domestic 

SATCOM 

1975 

2 

U.S.  Domestic 

COMSTAR 

1976 

3 

U.S.  Domestic 

MARI SAT 

1976 

3 

Maritime 

ANIK  A 

1972 

3 

Canadian  Domestic 

ANIK  B 

1978 

1 

Canadian  Domestic 

PALAPA 

1977 

2 

Indonesian  Domestic 

STATS lONAR 

1975 

2 

U.S.S.'r.  Domestic 

In 

Process 

Satellite  Series 

Number  Planned 

Usage 

INTELSAT  V 

8 

International 

WESTAR 

1 

U.S.  Domestic 

ADVANCED  WESTAR 

2 

U.S.  Domestic 

SATCOM 

1 

U.S.  Domestic 

SBS 

3 

U.S.  Domestic 

ANIK  C  and  D 

4 

Canadian  Domestic 

PALAPA 

3 

Indonesian  Domestic 

I  NS  AT 

2 

Indian  Domestic 

ECS 

1 

European  Regional 

MARECS 

2 

European  Maritime 

CS 

1 

Japanese  Domestic 

STATS lONAR 

6 

U.S.S.R.  ")omestic 

LOUTCH 

7 

U.S.S.R.  Domestic 

VOLNA 

7 

U.S.S.R.  Domestic 

Planned 

Satellite  Series 

Usage 

INTELSAT  VI 

International 

ARABSAT 

Arabian  Regional 

NORDSAT 

Nordic  Regional 

AT&T 

U.S.  Domestic 

BRASILSAT 

Brazilian  Domestic 

SATCOL 

Andean  Regional 

CHINASAT 

Chinese  Domestic 

ITALSAT 

Italian  Domestic 

AUSTRALIASAT 

Australian  Domestic 

ESTEC 

European  Regional 

HUGHES 

U.S.  Domestic 

SOUTHERN  PACIFIC 

U.S.  Domestic 

A  third  class  of  commercial 
satellites  that  are  not  included 
in  Table  1  are  direct  broadcast 
satellites.  To  date,  only  a  few 
satellites  of  this  type  have  been 
placed  into  service,  and  these 
have  been  primarily  of  an  experi¬ 
mental  nature.  However,  satel¬ 
lites  of  this  type  are  in  the 
planning  stage,  and  such  services 
will  be  provided  in  the  near  fu¬ 
ture  by  satellites  developed  by 
Japan,  Germany,  Canada,  the  Soviet 
Union,  the  European  Space  Agency, 
and  COMSAT. 

Satellite  Complexity  and  Micro¬ 
electronic  Usage 

The  proliferation  of  communi¬ 
cations  satellites  and  increasing 
use  of  their  services  have  re¬ 
sulted  in  additional  complexity 
due  to  the  need  for  greater  cir¬ 
cuit  capacities  and  more  commands, 
transponders,  beams,  and  fre¬ 
quencies.  One  measure  of  design 
complexity  is  the  number  of  indi¬ 
vidual  electronic  parts  on  a 


satellite.  Table  2  (from  Refer¬ 
ence  2)  shows  the  increased  parts 
count  and  telephone  circuit  capac¬ 
ity  for  the  INTELSAT  series  of 
satellites.  Parts  count  has  in¬ 
creased  from  3,500  for  INTELSAT  I 
to  19,000  in  the  current  opera¬ 
tional  INTELSAT  IV  and  IV-A.  Cir¬ 
cuit  capacity  is  25  times  as  great 
per  satellite.  The  INTELSAT  V 
satellites,  the  first  of  which  is 
planned  to  be  launched  and  put 
into  operation  later  this  year, 
will  have  over  54,000  electronic 
parts  and  will  double  the  present 
circuit  capacity  per  satellite. 

Domestic  communications  sat¬ 
ellite  complexities  are  also  in¬ 
creasing.  One  of  the  currently 
operating  U.S.  domestic  satel¬ 
lites,  Satcom,  has  about  18,000 
electronic  parts.  The  Insat  do¬ 
mestic  satellite  will  have  over 
36,000  parts.  Estimated  part 
counts  for  some  of  the  currently 
operating  and  planned  satellites 
are  shown  in  Table  3 . 


Table  2.  INTELSAT  Complexity 


No.  Parts  No.  Parts  Circuit  Circuit 

INTELSAT  No.  in  Transponders  Capacity  Capacity 

Satellites  Orbit  satellite^  Series  Series  per  per 

Satellite  Series 


Satellite 


No.  Electronic  Parts 


No.  Microcircuits 


INTELSAT  IV 

17,000 

1,700 

INTELSAT  IV-A 

19,000 

1,700 

MARI SAT 

15,000 

1,200 

SATCOM 

18,000 

1.800 

INTELSAT  V 

54,000 

4,000 

SBS 

30,000 

2,500 

INSAT 

36,000 

3,000 

Microelectronics  usage  has 
also  been  increasing  as  satellites 
become  more  complex.  The  first 
three  INTELSAT  satellites  were 
relatively  simple  and  made 
limited  use  of  microelectronics. 
INTELSAT  IV  and  IV-A  used  about 
1700  microcircuits  per  satellite, 
primarily  in  the  attitude  control, 
telemetry  and  command  systems. 
These  were  primarily  low-power  TTL 
SSI  types.  INTELSAT  V  will  use 
about  4,000  microcircuits  per  sat¬ 
ellite,  with  most  items  being  TTL 
SSI  and  MSI  types  used  in  the  at¬ 
titude  control  and  telemetry, 
tracking  and  command  systems.  In 
addition,  there  are  over  400  cus¬ 
tom  hybrid  circuits  used  in  DC- 
to-DC  converters  throughout  the 
satellite. 

In  most  of  the  domestic  sat¬ 
ellites  about  10  percent  of  the 
parts  have  been  microcircuits, 
primarily  digital  SSI  and  MSI 
types  in  the  attit’ude  control  and 
telemetry  and  command  systems . 
Almost  all  were  low-power  TTL 
types  except  for  the  use  of  CMOS 
in  Satcom.  Several  of  the  new 
satellites  plan  the  use  of  large 
hybrid  microcircuits  and  LSI 
microcircuits  such  as  microproces¬ 
sors,  memories,  and  custom  de¬ 
vices.  Insat  will  use  a  bipolar 
microprocessor  based  attitude  con¬ 
trol  system  with  44  LSI  microcir¬ 
cuits.  SBS  and  ANIK  C  will  use 
large  hybrid  microcircuits  for 
power  monitoring  functions  in 
their  telemetry  system.  Advanced 


Westar  plans  to  utilize  LSI  de¬ 
vices  in  attitude  control  and  TDMA 
switching  applications. 

Future  Microelectronic 
Applications 


The  continued  growth  and  suc¬ 
cess  of  commercial  communications 
satellites  depends  upon  the  use  of 
new  and  improved  techniques  which 
will  provide  additional  capacities 
and  services,  better  performance, 
greater  flexibility  and  control, 
and  continued  high  reliability. 

The  greater  use  of  microelectron¬ 
ics  can  significantly  contribute 
to  these  goals.  However,  the  use 
of  more  of  the  same  microelec¬ 
tronic  devices  that  are  in 
present-day  operational  satellites 
would  provide  an  unattractive 
approach.  The  use  of  state-of- 
the-art  devices  including  micro¬ 
processors  and  other  LSI  devices 
is  necessary  to  provide  additional 
capabilities  without  significantly 
increasing  the  number  of  parts 
required . 

There  are  a  number  of  space¬ 
craft  applications  in  which  commu¬ 
nications  satellite  performance 
could  be  improved  by  the  use  of 
state-of-the-art  microelectronics . 
Many  of  these  applications  are 
part  of  present  exploratory  re¬ 
search  and  development  programs 
and  include  attitude  control,  com¬ 
munications,  and  telemetry  and 
command  functions  as  shown  in 
Table  4. 
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Table  4.  Microelectronic  Spacecraft  Applications 


Attitude  Control 

Skewed  Reaction  Wheel  System 

Communications 

SS-TDMA  Switching 

Regenerative  Repeater 

Baseband  Signal  Processing 

Antenna  Beam  Reconfiguration 

•  •*  '« 

Telemetry  and  Command 

Central  Onboard  Processor 

Nv-'.v 

Telemetry  Processor 

V 

Power 

Battery  Conditioning 

Power  Management 

A  microprocessor  based  skewed 
reaction  wheel  attitude  control 
system  has  been  developed  which 
provides  greater  accuracy,  faster 
reaction  time,  and  higher  reliabil¬ 
ity.^  It  is  capable  of  maintain¬ 
ing  roll,  pitch,  and  yaw  accura¬ 
cies  of  better  than  0.05  degrees 
under  all  modes  of  operation.  The 
control  logic  unit  of  the  system 
uses  an  S-bit  bipolar  microproces¬ 
sor,  a  4K  PROM,  and  a  0.5K  RAM  and 
provides  double  precision  arithme¬ 
tic  for  attitude  control,  redun¬ 
dant  operation  of  the  reaction 
wheels,  and  telemetry  and  command 
inputs  and  outputs.  Add  time  is 
about  7  MS  and  multiply  time  is 
about  9  us.  The  predicted  system 
reliability  is  0.99  for  seven 
years . 

A  number  of  communications 
system  improvements  are  being 
developed  to  increase  overall  ef- 
efficiency  and  capacity.  These 
improvements  include  the  use  of 
complex  multibeam  antennas,  satel¬ 
lite  switching  of  time-division 
multiple-access  (TDMA)  beams  at 
microwave  frequencies,  regenera¬ 
tive  repeaters,  and  baseband  sig¬ 
nal  processing.  The  use  of  state- 
of-the-art  microelectronics  will 
be  a  key  element  in  the  uti 
tion  of  the  last  three  techn  _  s. 

Most  present-day  operational 
communications  satellites  employ 
frequency  modulation  and 
frequency-division  multiple-access 
(FDMA)  techniques  for  communica¬ 
tions  transmission.  In  TDMA  sys¬ 
tems,  a  given  channel  is  shared  by 
synchronizing  a  number  of  digital 
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transmissions  so  that  they  appear 
sequentially  at  the  input;  this 
was  first  introduced  operationally 
in  the  ANIK  system  in  1975.  Al¬ 
most  all  future  communications 
satellites  will  utilize  TDMA  tech¬ 
niques  to  more  efficiently  use 
bandwidth  and  power.  Satellite- 
switched  TDMA  (SS-TDMA)  is  a 
rapid,  dynamic  method  of  changing 
the  interconnections  between  dif¬ 
ferent  up-link  and  down-link  beams 
to  provide  additional  flexibility 
which  further  improves  efficiency. 

An  onboard  switching  center, 
which  has  been  developed  to  pro¬ 
vide  SS-TDMA,  contains  a  microwave 
switch  matrix  (MSM)  and  a  program¬ 
mable  distribution  control  unit 
(DCU).**  The  MSM  can  be  built  us¬ 
ing  pin  diodes  or  FET  transistors 
for  small  matrices  and  will  re¬ 
quire  microwave  monolithic  inte¬ 
grated  circuits  (MMIC)  for  large 
matrices.  The  DCU  can  include 
custom  LSI  circuitry,  microproces¬ 
sors,  and  LSI  memories.  It  in¬ 
cludes  a  telemetry  and  control 
interface  which  operates  at  slow 
speeds  (1  kHz)  and  control  elec¬ 
tronics  which  operate  at  the  high 
speeds  (6  ms)  required  to  switch 
the  matrix  elements. 

The  use  of  regenerative  re¬ 
peaters  is  being  explored  to  im¬ 
prove  satellite  transmission 
strength  and  reduce  power  require- 
mer*s  in  earth  stations.  Refer¬ 
ence  5  describes  the  potential  use 
of  this  technique.  The  transpon¬ 
der  demodulates  the  up-link  signal 
to  baseband  and  provides  onboard 
switching,  reformatting,  retiming. 
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and  remodulation  before  transmit¬ 
ting  the  signal  to  the  down-link. 

Demodulating  and  regenerating  the 
up-link  signal  of  the  satellite 
decouples  the  up-link  noise 
from  the  down- link  and  improves 
carrier-to-noise  ratio  and, 
therefore,  effective  signal 
strength.  The  system  utilizes 
microwave  integrated  circuits  for 
up-converters  and  down-converters 
and  an  intelligent  onboard  base¬ 
band  processor.^  This  processor 
includes  a  custom  LSI  circuit  or 
microprocessor  for  timing,  switch¬ 
ing,  and  control;  and  RAMs  and 
ROMs  to  store  the  regenerated  sig¬ 
nals.  The  onboard  baseband  pro¬ 
cessor  can  also  be  used  to  provide 
message  switching,  multiplexing, 
network  and  link  control,  traffic 
assignment,  buffer  management, 
digital  speech  interpolation,  and 
other  functions  now  performed  by 
computers  at  the  earth  stations. 

The  telemetry  and  command 
systems  of  communications  satel¬ 
lites  provide  many  housekeeping 
functions  including  command  decod¬ 
ing,  telemetry  processing,  power 
management,  battery  management, 
and  thermal  management.  The  pres¬ 
ent  approach  employs  a  separate 
component  to  provide  each  of  the 
required  monitoring  or  processing 
functions,  uses  redundancy  to 
improve  reliability,  and  provides 
control  by  commands  from  ground 
telemetry,  tracking,  and  command 
computers.  This  could  be  per¬ 
formed  by  an  onboard  microproces¬ 
sor  based  computer  as  described  in 
Reference  7.  It  would  include  an 
LSI  microprocessor  and  RAM  and  ROM 
memories.  Another  approach  is  a 
telemetry  system  which  continu¬ 
ously  samples,  conditions,  and  en¬ 
codes  measurements  of  satellite 
status,  attitude,  and  performance 
to  provide  control  and  fault  iso¬ 
lation.  This  system,  which  would 
provide  more  flexibility  and  im¬ 
proved  accuracy  and  reliability, 
would  include  custom  LSI  cir¬ 
cuits,^  microprocessor  bit  slices, 
PROMs,  RAMs  and  custom  hybrid 
circuits . 


General  Considerations 

There  are  numerous  applica¬ 
tions  in  which  state-of-the-art 
LSI  microelectronics  can  improve 
and  expand  communications  satel¬ 
lite  use  and  capabilities.  How¬ 
ever,  in  th^  selection  of  devices 
for  specific  applications,  several 
important  factors  should  be  con¬ 
sidered  including  the  speed, 
power,  environmental,  and  relia¬ 
bility  requirements. 

Figure  1  indicates  power  and 
speed  requirements  for  the  space¬ 
craft  systems  which  contain  the 
primary  applications  for  micro¬ 
electronics.  Telemetry  and  com¬ 
mand  normally  utilize  the  least 
power  and  require  the  slowest 
operation  rate.  Typically,  the 
telemetry  electronics  operates  at 
bit  rates  of  1000  bit/s  and  uses 
15  to  20  W  of  power.  Attitude 
control  electronics  usually  re¬ 
quire  25  to  30  W  of  power  and 
operating  speeds  of  about 
25  kbit/s.  The  communications 
system  applications  utilize  the 
largest  amounts  of  power  and  re¬ 
quire  speeds  of  10  to  500  Mbit/s 
for  onboard  signal  processing,  re¬ 
generative  repeaters,  and  SS/TDMA. 
Slow-speed,  low-power  applications 
are  best  suited  for  CMOS  types  of 
microcircuits;  bipolar  types  would 
be  required  for  the  high-speed, 
highr-power  applications  based  upon 
today ' s  technology . 


POWER 


Figure  1 .  Power  and  Speed 
Requirements 

The  primary  environmental 
consideration  relative  to  the  use 
of  microelectronics  is  radiation 
sensitivity.^  The  radiation  dose 
that  accumulates  during  seven 
years  in  a  geosynchronous  orbit 
for  silicon  shielded  by  aluminum 
or  lead  is  shown  in  Figure  2  (from 
Reference  10).  The  total  radia¬ 
tion  absorbed  reaches  a  point  be¬ 
yond  which  it  cannot  be  reduced  by 
further  shielding.  This  value  is 
3  X  10**  rads  for  lead  and  4  x  10^ 
rads  for  aluminum.  If  the  elec¬ 
tronics  are  placed  in  the  center 
of  a  satellite,  the  structure  and 
other  components  provide  approxi¬ 
mately  4  to  5  mm  of  aluminum 
shielding,  reducing  the  accumu¬ 
lated  7-yr  dose  on  a  microelec¬ 
tronic  device  to  between  10**  and 
10®  rads. 


The  increase  in  the  number  of 
years  required  is  also  shown  for 
added  thicknesses  of  aluminum 
shielding.  A  device  with  a  radia¬ 
tion  tolerance  of  10®  rads  will 
not  survive  two  years  even  with 
10  mm  of  shielding.  A  device  with 
10**  rads  radiation  tolerance  re¬ 
quires  about  6  itun  of  shielding  to 
survive  seven  years.  Most  cur¬ 
rently  popular  commercial  NMOS 
microprocessors  exhibit  failure 
between  10®  and  3  x  10®  rads.^^ 
Various  tests  on  CMOS  devices  have 
indicated  a  radiation  tolerance  of 
10**  to  10®  rads,  while  bipolar  de¬ 
vices  have  consistently  demon¬ 
strated  radiation  tolerances  of 
10®  rads  or  better. 
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Figure  2.  Dose-Depth  Curve  for 
A1  and  Pb  in  Geostationary 
Orbit 

Figure  3  (also  from  Refer¬ 
ence  10)  shows  the  number  of  years 
required  for  a  device  with  4.5  mm 
of  shielding  to  reach  four  differ¬ 
ent  assumed  fatal  radiation  doses. 
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Figure  3.  Survival  of  CMOS 

Circuits  in  a  Particular 
Location  of  a  Conununi- 
cations  Satellite, 

The  performance  of  commercial 
communications  satellites  has  been 
characterized  by  very  high  reli¬ 
ability.  The  continuity  of  serv¬ 
ice  of  the  INTELSAT  space  segment 
has  exceeded  0.99995  since  1970. 
This  high  reliability  has  been 
maintained  by  reliability  programs 
incorporating  key  elements  such  as 
redundancy,  the  use  of  devices 
with  suitable  design  lifetime,  and 
the  elimination  of  potentially  de¬ 
fective  electronic  parts  through 
testing,  screening,  and  burn-in. 

This  high  reliability  has 
continued  even  though  satellite 
complexity  has  increased.'  This 
has  been  due,  in  part,  to  the  use 
of  SSI  and  MSI  microelectronic  de¬ 
vices.  These  microcircuits  allow 
the  integration  of  a  number  of 
discrete  parts,  and  therefore, 
more  functions  were  included  with¬ 
out  a  corresponding  increase  in 
parts.  In  the  future,  this  trend 
can  be  continued  by  using  LSI  de¬ 
vices.  In  fact,  additional  reli¬ 
ability  improvement  should  be  pos¬ 
sible  due  to  a  smaller  number  of 
physical  parts  per  function,  the 
elimination  of  interconnections, 
and  the  availability  of  added 
flexibility  for  redundancy. 

The  traditional  method  of  de¬ 
signing  spacecraft  has  been  to 
build  components  for  particular 
tasks  and  to  redundantly  cross 
couple  multiple  components  in  par¬ 
allel  to  improve  reliability.  By 
replacing  certain  components  with 
general  purpose  microprocessor 
modules,  this  same  method  of  re¬ 
dundancy  can  be  continued.  In  ad¬ 
dition,  these  microprocessor  mod¬ 
ules  can  perform  numerous  tasks, 
and  any  task  can  be  shared  between 
a  number  of  microprocessor  mod¬ 
ules.  By  grouping  the  micropro¬ 
cessors,  each  microprocessor  could 
then  perform  its  own  task,  as  well 
as  be  able  to  take  over  other 
tasks.  Therefore,  reliability 
would  be  further  Improved. 


The  design  lifetime  of  LSI 
microelectronic  devices  should  not 
normally  significantly  affect  sat¬ 
ellite  reliability.  The  normal 
degradation  of  a  typical  semicon¬ 
ductor  device  is  such  that  a  mean- 
time-to-failure  of  at  least  one 
million  hours,  or  greater  than  one 
hundred  years,  can  be  predicted. 
Even  though  an  LSI  device  contains 
a  large  number  of  semiconductor 
junctions,  the  mean-time-to- 
failure  should  still  be  much 
greater  than  required.  The  only 
factor  that  could  limit  the  design 
lifetime  is  failure  due  to  low  ra¬ 
diation  tolerance. 

Reliability  programs  for 
long-life  commercial  satellites 
have  included  screening  programs 
to  eliminate  potentially  defective 
electronic  pav,.s.  Parts  programs 
have  required  qualification,  con¬ 
trol,  and  derating  of  all  elec¬ 
tronic  parts.  Typical  INTELSAT 
part  screening  procedures  have  in¬ 
cluded  burn-in  of  all  active  elec¬ 
tronic  parts  for  672  hr  with  a 
minimum  of  four  data  points  which 
are  analyzed  for  parameter  drift. 
These  reliability  procedures 
should  be  applied  to  LSI  devices 
used  in  future  satellites.  In  ad¬ 
dition,  lot  radiation  qualifica¬ 
tion  may  be  required.  Also,  since 
the  devices  are  more  complex  in 
both  function  and  design,  adequate 
functional  testing  must  be  devel¬ 
oped  and  utilized.  In  addition  to 
AC  and  DC  parametric  testing,  test 
programs  must  be  developed  to  ver¬ 
ify  that  as  many  faults  as  possi¬ 
ble  have  been  identified. 

Future  satellite  microelec¬ 
tronic  applications  will  utilize 
standard  microprocessor  and  LSI 
devices  and  custom  LSI  devices. 
Standard  microprocessors  are  gen¬ 
eral  purpose  devices  that  can  be 
used  to  digitally  process  most 
tasks.  Custom  LSI  devices  are 
special  purpose  units  which  can  be 
designed  to  perform  digital  tasks 
similar  to  those  of  a  microproces¬ 
sor  or  analog  or  a  mixture  of  ana¬ 
log  and  digital  functions.  Both 
types  of  devices  have  potential 
applications  in  satellites.  Al- 


though,  in  some  cases,  either  can 
be  used,  in  terms  of  reliability, 
the  "standard"  microprocessor  is 
preferable.  First  of  all,  the 
microprocessor  is  a  general  pur¬ 
pose  device  and  can  provide  added 
redundancy  for  multiple  tasks  as 
previously  discussed.  In  addi¬ 
tion,  the  complexity  of  the  LSI 
device  increases  the  difficulty  of 
testing.  The  probability  of  de¬ 
veloping  a  test  program  to  find 
and  correct  design  errors  depends 
on  the  number  of  devices  tested. 
Microprocessors  are  produced  in 
volume  for  various  applications. 
Satellite  applications  for  special 
purpose  custom  LSI  chips  will 
normally  require  a  small  number  of 
devices,  and,  therefore,  test  pro¬ 
grams  for  such  devices  will  not  be 
as  effective.  Finally,  it  is  dif¬ 
ficult  to  obtain  high-quality  man¬ 
ufacturers  who  will  produce  low- 
volume  custom  LSI  devices.  Those 
who  will  produce  such  devices  nor¬ 
mally  use  MOS  techniques  which 
presently  have  a  low  radiation 
tolerance.  Bipolar  and  possibly 
CMOS  standard  microprocessors  are 
available  which  have  a  higher  ra¬ 
diation  tolerance. 

Conclusions 

The  use  of  state-of-the-art 
LSI  microelectronic  devices  is  re¬ 
quired  in  future  commercial  commu¬ 
nications  satellite  applications 
to  fulfill  the  need  for  added  com¬ 
plexity  while  maintaining  high  re¬ 
liability.  Both  standard  micro¬ 
processor  and  LSI  devices  and  cus¬ 
tom  LSI  chips  can  be  used,  but 
microprocessors  are  preferred  if  a 
choice  is  possible.  Important 
considerations  in  the  selection 
and  application  of  LSI  devices  in¬ 
clude  speed  and  power  require¬ 
ments,  radiation  tolerance,  and 
reliability  factors  such  as  redun¬ 
dancy,  testability,  availability, 
and  capability. 
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CLASS  S  LSI  REQUIREMENTS. 
CURRENT  STATUS, 

AND  PROJECTIONS 
FOR  THE  FUTURE 

by 

James  J.  Egan 

Director,  Components  and  Product 
Engineering  Department 
The  Aerospace  Corporation 

SUMMARY 

Class  S  requirements  for  LSI  circuits 
are  currently  in  a  very  preliminary 
stage.  Existing  LSI  specifications, 
and  associated  quality  control  pro¬ 
cedures  and  test  methods,  are 
essentially  an  extension  of  those 
employed  for  SSI  and  MSI  and  do  not 
reflect  the  significant  differences 
encountered  when  dealing  with  LSI. 

For  example,  many  screening  tech¬ 
niques  such  as  precap  visual  inspect¬ 
ion,  electrical  test  or  burn-in  which 
were  relied  upon  heavily  in  the  past  to 
achieve  reliable  integrated  circuits 
cannot  be  as  effectively  applied  to  LSI. 
Proposed  MJL-STD-1547  contains 
sections  specifically  covering  LSI; 
however,  these  are  largely  undevel¬ 
oped.  In  recognition  of  the  new 
procurement  and  testing  problems 
presented  by  LSI,  the  Air  Force  Space 
Division's  Space  Parts  Working  Group 
has  formed  a  special  LSI  committee 
to  study  and  make  recommendations  in 
the  areas  of  General  Requirements, 
Certification  and  Qualification,  Testing 
and  Characterization,  and  Radiation 
Hardness  for  Class  S  LSI  circuits. 
Major  problems  which  are  currently 
inhibiting  the  growth  of  LSI  applications 
in  Space  Division  programs  include 
radiation  hardness,  adequate  reli¬ 
ability  testing  and  the  high  cost  assoc¬ 
iated  with  the  development  and  manu¬ 
facture  of  small  quantities  of  special 
circuits.  In  order  to  effectively 
achieve  Class  S  LSI  for  future  pro¬ 
grams,  it  will  be  necessary  to: 

1)  Develop  realistic  and 
consistent  performance 
and  radiation  require¬ 
ments. 


2)  Establish  qualified,  radiation 
hard  LSI  technologies. 

3)  Establish  qualified,  standard 
computer  aided  design  systems 
for  custom  LSI  circuits. 

4)  Establish,  radiation  hard 
standard  LSI  circuits. 

5)  Evolve  a  more  appropriate 
set  of  product  assurance 
requirements  and  test  methods, 

CONCLUSIONS 

o  Existing  Class  S  microcircuit 
requirements  are  inadequate 
for  LSI. 

o  Custom  LSI  circuits  will  offer 
significant  advantages  to 
future  Space  Division  projects. 

o  Radiation  and  performance 

requirements  limit  the  use  of 
LSI  technology. 

o  Standard  LSI  building  blocks 

will  be  essential  to  all  systems 

o  LSI  testability  is  the  major 
reliability  problem. 

RECOMMENDATIONS 

o  Develop  a  set  of  product 

assurance  requirements  and 
test  methods  which  are  eff¬ 
ective  for  LSI. 

o  Develop  qualified  standardized 
computer  aided  LSI  design 
systems  and  associated  rad¬ 
iation  hard  LSI  technologies. 

o  Establish  realistic  and  con¬ 
sistent  radiation  and  perform¬ 
ance  requirements. 

o  Establish  a  qualified  radiation 
hard  set  of  LSI  building  blocks, 

o  Implement  design  for  test¬ 
ability  requirements  in  all 
future  custom  LSIC's. 


Status  of  Class  S  LSI  Requirements 


Solutions  to  LSI  Radiation  Problem 


o  No  essential  differences  from 
SSI/MSI  defined  as  yet, 

o  Proposed  MIL-STD-1547  con¬ 
tains  LSI  sections  which  are 
very  preliminary. 

o  USAF/SD  Space  Parts  Working 
Group  has  formed  a  committee 
to  make  recommendations. 

o  Existing  requirements  for  Class 
S  microcircuits  not  adequate 
for  LSI. 

Problems  with  Applying  SSI/MSI 

Requirements  to  LSI 

o  Characterization  typically 
Incomplete. 

o  Exhaustive  functional  testing 
not  feasible. 

o  Device  characteristics  not 
measurable, 

o  Visual  chip  inspection  in¬ 
effective. 

o  Operating  burn-in  not  readily 
implemen  table, 

o  Fault  location  (failure  analysis) 
very  costly. 

o  Special  in-process  controls 
more  difficult  to  institute. 

Major  Obstacles  to  Class  S  LSI 

o  Testing,  Functional  and  Device. 

o  Failure  experience. 

o  Radiation  Hardness. 

o  Special,  low  volume  require¬ 
ments. 

Solutions  to  LSI  Testing  Problem 

o  Thorough  characterization  of 
standard  LSI  functional  building 
blocks. 

o  Incorporation  of  reliability  test 
chips  on  LSI  wafers. 

o  Design  for  testability  on  custom 
LSI  circuits. 

o  Sharing  of  LSI  test  programs 
and  results  among  Class  S  users. 


o  Establish  realistic  and  con¬ 
sistent  radiation  requirements. 

o  Consider  maximum  effective 
use  of  system  design  and 
shielding  alternatives. 

o  Establish  qualified,  standard 
radiation  hard  technologies. 

Solutions  to  the  LSI  Reliability 

Problem 

o  Establish  qualified,  standard 
functional  building  blocks. 

o  Establish  qualified,  standard 
computer  aided  design  systems 
and  design  centers, 

o  •  Establish  qualified,  standard 
LSI  technologies  and  manu¬ 
facturing  facilities. 

o  Develop  LSI  specific  quality 
control  methods, 

o  Share  LSI  reliability  test  and 
failure  experience  data  among 
Class  S  users. 

Concluding  Observations 

o  Class  S  LSI  will  be  largely 
"custom"  due  to  testability, 
radiation  hardness,  and  quality 
control  requirements. 

o  Considerable  "up-front"  effort 
will  be  required  to  establish 
standard,  qualified  building 
blocks,  CAD  systems,  and 
radiation  hard  technologies, 

o  A  much  more  intensive  and 
coordinated  research  and  de¬ 
velopment  effort  is  required  to 
meet  the  needs  of  the  space 
community. 


.f 
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SPACE  RADIATION  EFFECTS  AND  COSMIC  RAY 
INDUCED  UPSETS  IN  MODERN  SEMICONDUCTOR  DEVICES* 

William  E.  Price 
Jet  Propulsion  Laboratory 
Pasadena,  California 


INTRODUCTION 

Electronic  systems'  in  spacecraft  and 
satellites  can  be  hardened  against 
space  radiation  effects.  The  hardening 
effort  must  be  started  early  in  the 
system  design  to  keep  costs  within 
acceptable  limits.  System  hardening 
is  mostly  a  parts  selection  problem 
interwoven  with  proper  circuit  design 
backed  by  shielding  analysis  and  ap¬ 
plication  for  space  radiation  situa¬ 
tions. 

Space  radiation  consists  mostly  of 
charged  particles  and  some  electro¬ 
magnetic  secondary  radiation.  The 
particles  have  a  vast  range  of  ener¬ 
gies,  most  of  which  are  not  highly 
penetrating.  Therefore,  the  intrinsic 
mass  of  the  spacecraft  has  value  as 
shielding  as  well  as  the  materials 
which  have  been  added  specifically 
for  such  purposes.  The  surface 
materials  of  a  spacecraft  receive  much 
higher  doses  than  the  interior  of  the 
spacecraft.  However,  the  semiconduc¬ 
tor  devices  are  orders  of  magnitude 
more  sensitive  than  any  material  or 
other  types  of  devices.  So  the 
system  hardening  problem  resolves 
into  a  program  of  semiconductor  device 
selection  and  circuit  design  to 
accommodate  the  radiation  effects  as 
determined  by  radiation  tests. 

Another  space  radiation  type  of  concern 
is  cosmic  rays  which  can  Induce  single 
event  upsets  in  memory  devices.  This 
problem  has  only  come  into  view  since 
1975  but  as  device  geometries  get 
smaller  this  problem  gets  larger.  Other 
radiation  types  will  not  be  considered 
here,  although  it  is  well  known  that 
gamma  rays  and  neutrons  from  nuclear 
weapons  are  highly  penetrating  and  can 


not  be  readily  shielded.  Also,  the 
rate  of  irradiation  can  be  an  important 
factor  for  nuclear  weapon  environment. 
However,  the  rate  of  irradiation  for 
most  space  radiation  is  low  enough  so 
that  there  is  no  problem  to  electronic 
systems. 

SELECTION  OF  SEMICONDUCTOR  DEVICES 

System  hardening  by  device  selection 
depends  to  some  extent  on  the  radiation 
dose  level  specified  for  the  mission. 
Levels  of  5000  rad (Si)  or  under  are 
relatively  less  difficult  than  the  dose 
levels  for  spacecraft  such  as  Voyager 
and  Galileo  which  must  survive  the 
severe  radiation  fields  at  Jupiter  and 
have  specification  levels  of  150  Kilo- 
rads(Si).  However,  the  approach  to 
hardening  is  similar. 

The  philosophy  in  the  approach  to 
selection  of  devices  is  the  result  of 
several  technical  facts  which  are  some¬ 
what  related  to  the  details  of  the 
radiation  environment  and  mission  para¬ 
meters.  It  is  important  to  keep  in 
mind  that  space  radiation  produces  two 
major  effects  in  semiconductor  devices; 
surface  ionization  effects  and  bulk 
damage.  The  surface  effects  are  the 
most  important  because  they  are  the 
most  prevalent  effect.  The  bulk  damage 
effects  become  important  only  for  high 
energy  particles  and  in  devices  which 
have  a  large  base  such  as  low  frequen¬ 
cy  pnp  transistors  (faj,  <100  MHz)  and 
power  devices. 

The  radiation  induced  surface  effects 
are  directly  related  to  the  physical 
conditions  of  the  device  surfaces  and 
the  oxide  layers  which  are  directly 
related  to  the  details  of  the  manufac¬ 
turing  processes  and  especially  the 


*Thls  paper  presents  the  results  of  one  phase  of  research  carried  out  at  the 
Jet  Propulsion  Laboratory,  California  Institute  of  Technology,  under  Contract 
No.  NAS7-100,  sponsored  by  the  National  Aeronautics  and  Space  Administration. 
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high  temperature  steps.  Selection  of 
devices  for  total  dose  hardened  systems 
Is  dependent  on  characterizing  devices 
with  radiation  tests  which  represent 
a  process  line  and  a  particular  manu¬ 
facturer.  This  means  that  radiation 
effects  determined  for  one  device  type 
and  one  manufacturer  is  not  valid  for 
all  manufacturers  of  that  type. 

For  example,  it  has  been  found  that 
there  may  be  as  much  as  three  orders 
of  magnitude  difference  in  sensitivity 
between  two  manufacturers  of  the 
2N2222  transistors. 

In  addition  to  this  problem  it  also  has 
been  shown  that  there  exist  serious 
differences  between  diffusion  lots  and 
between  wafers  in  the  same  diffusion 
lot.  These  differences  occur  because 
of  process  step  differences  uncontrol¬ 
led  in  manufacture.  Radiation  in¬ 
duced  surface  ionization  effects  can¬ 
not  be  predicted  by  any  pre-irradia¬ 
tion  electrical  or  physical  measure¬ 
ments.  As  a  result,  all  devices  must 
be  radiation  tested  during  the 
design  phase  of  the  system  or  else 
relatively  recent  test  data  must  be 
found  which  characterizes  the  desired 
electrical  parameters  at  the  speci¬ 
fication  radiation  levels.  It  is 
useful  to  have  data  both  above  and 
below  the  specified  dose  level  to 
allow  a  determination  of  the  safety 
margin  and  the  requirements,  if  any, 
for  shielding  the  part.  A  method  for 
doing  total  dose  radiation  tests  has 
been  developed  (see  Ref.  5). 

After  devices  have  been  found  adequate 
and  designed  into  the  system  there  is 
a  production  phase  in  which  one  or  more 
of  the  systems  are' built.  The  produc¬ 
tion  may  occur  soon  after  the  design 
or  years  later.  Hardness  assurance 
practices  require  radiation  testing 
of  a  sample  from  each  lot  of  semicon¬ 
ductor  parts  purchased  unless  it  can 
be  shown  that  there  is  an  adequate 
safety  factor  which  precludes  testing. 
It  has  been  common  to  find  that  when  a 
long  period  has  passed  (after  the 
radiation  test  data  was  obtained  and 
used  in  design)  until  the  production, 
that  the  data  base  can  slip  and  may 
even  impact  design.  For  this  reason 
it  is  prudent  to  use  a  sufficelnt 
safety  margin  during  design. 


The  Galileo  spacecraft  uses  a  safety 
margin  of  two  for  engineering  sub¬ 
systems.  This  spacecraft  is  being 
engineered  at  the  Jet  Propulsion 
Laboratory  to  be  launched  in  1985. 

It  will  orbit  Jupiter  a  number  of 
times.  Table  1  shows  the  transistors 
which  were  radiation  tested  for  use  on 
the  spacecraft.  The  third  column 
shows  which  of  the  device  types  had 
hardness  assurance  requirements. 
Comparison  of  column  2  and  3  shows 
the  rejection  rate.  Table  2  shows  a 
similar  array  of  information  for  In¬ 
tegrated  circuits  such  as  operational 
amplifiers,  comparators,  and  others. 

Table  3  shows  all  LSI  devices  tested. 
None  of  the  LSI  devices  were  subjected 
to  HA  accept/reject  criteria.  In  fact, 
while  all  of  the  LSI  devices  were  test¬ 
ed  to  find  out  if  they  were  useful  for 
Galileo,  most  were  found  to  be  too 
sensitive  for  consideration.  The 
AD571  is  an  exception  as  well  as  the 
1800  series  parts.  These  are  present¬ 
ly  being  considered  for  Galileo.  It 
is  well  established  that  the  trend 
coward  smaller  geometry  and  larger 
complexity  in  LSI  type  devices  results 
in  greater  radiation  sensitivity. 
Complex  LSI  devices  are  failing  at 
dose  levels  between  1  and  75  Kilorads 
(Si).  For  more  details  on  radiation 
hardening  of  systems  for  space  appli¬ 
cations  see  References  1  through  4.- 
A  guideline  document  is  being  prepared 
concerned  with  methods  used  for  ob¬ 
taining  hardness  assured  devices  (See 
Ref.  6).  A  flow  diagram.  Figure  1, 
from  that  document  gives  a  design  and 
hardness  assurance  plan  for  hardening 
systems. 

Single  Event  Upsets  from  Cosmic  Rays 

A  relatively  new  phenomenon  has  come 
to  notice  in  recent  years  which  can, 
and  has,  caused  problems  with  oper¬ 
ational  space  systems.  Cosmic  ray 
particles  have  been  shown  to  induce 
soft  errors  in  memory  systems  and  in 
information  handling  systems. 

A  series  of  laboratory  tests  carried 
out  within  the  last  year  have  out¬ 
lined  the  problem  to  an  extent  not 
possible  by  theoretical  analysis 
alone.  Previously  analyses  of 


specific  device  types  had  determined 
that  devices  (being  used  in  operatio¬ 
nal  space  vehicles)  could  be  upset  by 
Ions  In  the  mass  region  of  Iron.  These 
Ions  are  plentiful  In  the  cosmic  ray 
spectrum.  The  laboratory  tests  have 
confirmed  the  theoretical  analyses  and 
have  gone  a  step  or  two  further.  A 
fair  array  of  device  types  already  In 
space  or  planned  for  use  In  space 
vehicles  were  exposed  to  150  MeV  Argon 
and  Krypton  Ions.  The  specific  ioni¬ 
zation  of  these  Ibns  bracket  that  of 
Iron. 

What  the  experiments  have  shown  Is 
that  the  memory  devices  in  common  use 
have  a  wide  range  of  susceptibility 
to  cosmic  ray  Induced  upset.  Dyna¬ 
mic  RAM's  are  particularly  suscepti¬ 
ble  because  the  capacitance  of  the 
sensitive  node  gets  quite  low  during 
refresh  cycles.  Certain  CMOS  memories 
have  been  shown  to  have  a  low  sensiti¬ 
vity  to  upset.  These  Include  memory 
device  types  used  in  the  Voyager  space¬ 
craft  and  In  the  Galileo  spacecraft. 

Tests  of  dynamic  devices  with  50  MeV 
protons  and  lA  MeV  neutrons  have  shown 
them  to  be  upset.  However,  a  similar 
test  of  a  large  number  of  static  memory 
devices  showed  no  upset.  The  mechanism 
for  upset  with  neutron  and  protons  is 
the  production  of  alpha  particles  with¬ 
in  the  memory  cell  by  nuclear  reactions. 
Modeling  of  the  sensitive  regions  of 
memory  devices  can  help  predict  whether 
a  particular  device  will  be  susceptible, 
but  there  are  some  unknown  quantities 
which  must  be  assumed  or  estimated. 

Also,  critical  Information  Is  often 
difficult  and  time  consuming  to  obtain 
from  the  manufacturer,  making  predict¬ 
ions  less  sure.  Ultimately  It  Is  still 
necessary  to  carry  out  a  test  to  assure 
the  validity  of  the  model.  So  that  It 
Is  recommended  that  tests  be  carried 
on  all  devices  which  are  to  be  consider¬ 
ed  for  use  on  spacecraft.  As  newer 
device  types  become  available  there  Is 
a  tendency  for  the  cell  sizes  to  be 


smaller.  This  will  likely  cause 
greater  susceptibility  to  single 
event  upsets.  The  problem  Is  bound 
to  become  greater  for  future  appli¬ 
cations  using  the  more  sophisticated 
new  device  type.  Error  correction 
systems  will  be  needed  to  compensate. 
References  5  through  13  give  a  full 
treatment  of  the  single  event  upset 
problem  as  induced  by  space  radiations. 
In  addition,  reference  13  gives  a  view 
of  the  expect  rate  of  upsets  from 
cosmic  ray  showers  for  earth  bound 
systems. 

Conclusions  and  Recommendations 

1.  An  array  of  discrete  semiconductor 
devices  can  be  procured  which  are 
suitable  for  use  In  a  radiation 
hardened  system.  It  Is  recommended 
that  for  Hardness  Assurance,  margi¬ 
nal  devices  should  be  lot  sample 
tested . 

2.  Large  Scale  Integration  (LSI  devices 
are  radiation  soft;  1  to  25  Kllorad 
(SI) .  Radiation  hardened  systems 
may  not  be  able  to  use  such  devices 
without  special  hardening  efforts, 
l.e.,  shielding  or  redesign. 

3.  Single  event  upsets  In  memory  devices 
induced  by  cosmic  rays,  and  other 
heavily  Ionizing  events  can  be  a 
problem  to  spacecraft  and  earth 
bound  systems.  It  will  be  a  contin¬ 
ually  worsening  problem  for  the  fu¬ 
ture  where  device  geometries  become 
ever  more  upset  sensitive  due  to 
their  decreasing  size.  It  Is 
recommended  that  all  memory  devices 
be  tested  to  confirm  their  upset 
susceptibility. 
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TABLE  1 


DISCRETE  SEMICONDUCTOR  DEVICES  RADIATION 
TESTED  FOR  THE  GALILEO  SPACECRAFT  PROGRAM 


DEVICE  NO. 

NO. OF  LOTS 
TESTED 

NO. OF  LOTS 
REJECTED 

DEVICE  NO. 

NO. OF  LOTS 
TESTED 

NO. OF  LOTS 
REJECTED 

CUD 

4 

0 

2N3251 

4 

3 

MQ2905 

2 

1 

2N3331 

1 

0 

SDT3303 

2 

0 

2N3350 

6 

2 

SD3304 

5 

1 

2N3391 

1 

0 

SD3323 

5 

3 

2N3467 

2 

0 

SDT5553 

1 

0 

2N3501 

1 

0 

2N918 

1 

0 

2N3637 

1 

0 

2N2060 

2 

0 

2N3700 

7 

1 

2N2222 

14 

5 

2N3799 

2 

0 

2N2369 

1 

0 

2N3805 

5 

N/A 

2N2484 

1 

N/A 

2N3821 

1 

N/A 

2N2608 

1 

N/A 

2N3824 

1 

N/A 

2N2658 

2 

1 

2N4044 

2 

1 

2N2880 

2 

0 

2N4260 

1 

N/A 

2N2905 

3 

1 

2N4856 

1 

0 

2N2907 

15 

9 

2N5087 

1 

N/A 

2N2920 

8 

5 

2N5196 

5 

3 

2N2975 

4 

N/A 

2N5556 

2 

N/A 

2N3032 

1 

N/A 

2N5663 

2N6138 

96SV131 

14BB101 
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1 

1 

3 

1 

0 

0 

0 

N/A 

- . 

TABLE  2 


INTEGRATED  CIRCUITS  RADIATION  TESTED 
FOR  THE  GALILEO  SPACECRAFT  PROGRAM 


DEVICE  NO. 

NO.  OF  LOTS 
TESTED 

NO. OF  LOTS 
REJECTED 

DEVICE  NO. 

NO.  OF 
TES 

CD4001 

1 

N/A 

HA2-2520 

2 

CD4011 

1 

N/A 

HA2700 

1 

CD4013 

3 

N/A 

Hl-2-200 

2 

CD4014 

1 

N/A 

H1-1800A-2 

2 

CD4027 

2 

N/A 

LF155 

1 

CD4049 

2 

N/A 

LF156 

2 

CD4052 

2 

N/A 

LMlOl 

18 

CD4053 

2 

N/A 

LM106 

1 

CD4066 

1 

N/A 

LM108 

40 

CD4099 

2 

N/A 

LMlll 

47 

CD40115D 

1 

N/A 

LM119 

20 

DG141 

1 

N/A 

LM139 

19 

HAl-2420 

2 

N/A 

LM158 

1 

HA2-2050-2 

1 

N/A 

4720DM 

1 

NO.  OF  LOTS 
REJECTED 


TABLE  3 

LSI  DEVICES  RADIATION  TESTED  FOR  THE 
GALILEO  SPACECRAFT  PROGRAM 


DEVICE  NO 


NO.  OF  LOTS 
TESTED 


DEVICE  NO. 


NO.  OF  LOTS 
TESTED 


AD7521 

ADC1210 

CCDS 00X8 00 

CDP1802 

CDP1834 

CDP1852 

CDP1856 


HM16611-9 

HM6508 

HM9-6551B-9 

IM6508MDE 

MB5101L-4 

MCM418M 


MM54C905 

MM54C920D 

MM54C929 

MM70C95 


MN5216H 

MN9181 

MP7570 

MWS5001D 

MWS5501 

SBP9900X 


TCC244 

TDCIOOI-J 

TDC1021-J 


54C200 
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Nuclear  Weapon  Radiation  Effects 


1.0  Introduction 

Five  years  ago,  the  radiation  effects  com¬ 
munity  was  very  confident  that  LSI  could  be 
hardened  to  levels  demanded  by  strategic 
weapon  systems  and  military  satellite  sys¬ 
tems.  The  Lockheed  C-U  parts  program  was 
well  underway  at  that  time,  and  it  was  ex¬ 
pected  that  the  parts  family  being  developed 
thereunder  would  represent  the  last  example 
of  military  user  being  many  years  behind  the 
commercial  world.  Two  major  process  tech¬ 
nologies  were  being  developed  for  hard  LSI 
applications  (viz.  CMOS/SOS  and  I  *L),  and 
both  were  judged  capable  of  replacing  and 
surpassing  dielectrically  isolated  LSTTL. 
Today,  the  only  parts  available  for  use  in 
systems  which  must  be  hardened  at  strategic 
levels  to  nuclear  weapons  effects  are  the 
C-U  parts  or  their  derivatives  developed 
under  the  MX  program.  Instead  of  being  five 
years  behind  the  commercial  world,  we  are 
ten  years  behind  and  expect  to  see  the  lag 
increase  for  some  time.  It  is  time  to  re¬ 
consider  our  strategy  for  the  development  of 
radiation  hardened  LSI.  This  paper  makes 
some  suggestions  for  a  new  strategy  for  at¬ 
tacking  this  problem. 

2.0  Statement  of  Problem 

2.1  Impact  of  Nuclear  Weapon  Effects  on 
LSI  Technology  Selection 

Contrary  to  popular  opinion,  neutron  damage 
effects  rarely  limit  the  application  of  LSI 
technologies  to  strategic  missile  systems  and 
never  limit  in  the  case  of  military  space  sys¬ 
tems.  Total  ionizing  radiation  dose  and  dose 
rate  are  the  elements  of  the  environment 
which  determine  what  can  and  can  not  be 
used.  To  date,  the  radiation  effects  com¬ 
munity  has  started  the  search  for  suitable 
technologies  with  the  assumption  that  the 
inherent  susceptibility  of  junction  isolated 
(31)  IC  processes  to  transient  ionizing  radia¬ 
tion  dictated  the  use  of  dielectrically  iso¬ 
lated  (DI)  processes.  Specific  DI  processes 
which  have  been  exploited  include  the  single 
crystal  bucket  approach  (for  bipolar)  and  si- 
licon-on-sapphire  (for  MOS).  Neither  of  these 
could  be  described  as  a  consensus  selection 
for  use  in  LSI,  but  they  are  free  of  radi¬ 


ation  induced  latchup,  and  resistant  to 
burnout  or  logic  upset  in  transient  ioni¬ 
zing  radiation.  Consequently  they  have  re¬ 
ceived  major  attention  for  hardened  LSI. 

2.2  Current  Status  of  Dielectrically 
Isolated  Integrated  Circuit 
Development 

Bipolar  DI  integrated  circuits  are  pre¬ 
sently  at  a  level  of  integration  correspon¬ 
ding  to  SSI/MSI.  Figure  1  lists  the  types 
of  digital  functions  one  can  find  available 
in  this  technology.  It  is  questionable  how 
much  further  one  can  go  in  level  of  integra¬ 
tion  with  such  an  approach.  MOS  integrated 
circuits,  on  the  other  hand,  never  really 
attempted  to  develop  through  the  SSI/MSl 
route.  CMOS/SOS  technologists  jumped  dir¬ 
ectly  to  LSI  in  their  attempts  to  produce 
hardened  integrated  circuits.  Differences 
of  opinion  exist  as  to  where  the  technology 
actually  stands  today,  but  it  is  safe  to 
that  no  major  strategic  or  space  systems 
plan  to  procure  radiation  hardened  CMOS/ 
SOS  LSI  for  use  in  flight  hardware  for 
at  least  two  years.  Progress  made  in  this 
area  has  not  been  up  to  the  expectations 
of  those  working  in  the  field. 

3.0  Possible  Alternate  Approaches 

Many  weapons  and  reconnaissance  systems 

to  be  deployed  during  the  1980's  require 
LSI  to  perform  their  functions  properly. 

If,  as  appears  to  be  the  case,  it  will 
not  be  possible  to  procure  DI  devices 
for  this  purpose  in  the  time  frame  of 
interest,  we  must  re-examine  the  utility 
of  what  we  can  procure.  Bipolar  bit- 
slice  processors,  memories,  and  gate 
arrays  are  available  in  militarized  ver¬ 
sions  of  commercial  devices  fabricated 
in  junction  isolated  technology.  Simi¬ 
larly,  there  is  good  evidence  that  a 
useful  range  of  CMOS  devices  on  bulk 
silicon  could  be  made  available  with 
some  additional  development  work  (Cf. 

Bill  Dawe's  discussion  in  this  workshop). 

If  system  level  approaches  are  taken 
to  transient  ionizing  radiation  har- 


dening,  either  or  both  of  these  techno-  This  can  be  done  periodically  (strobing)  or 

logies  could  provide  LSI  useful  for  on  demand  (crowbarring).  The  technique  has 

military  system  application.  Require-  been  used  on  memory  systems,  and  should  be 

ments  which  these  system  hardening  ap-  extendable  to  complete  LSI  systems, 

proaches  must  satisfy  can  now  be  dis¬ 
cussed.  4.0  Summary  and  Conclusions 

3.1  Requirements  for  System  Level  Development  of  dielectrically  isolated 

Hardening  of  LSI  Systems  LSI  hardened  to  nuclear  weapon  radia- 


Junction  isolated  IC's  respond  to  transient 
ionizing  radiation  by  passing  large  photo¬ 
currents.  Photocurrent  can  cause  destruc¬ 
tive  burnout  of  chips,  logic  upset,  or 
latchup.  To  harden  systems  against  these 
failure  modes,  it  is  necessary  to  limit  the 
current  which  a  chip  can  draw  from  the  sup¬ 
ply  during  the  prompt  pulse,  to  protect 
critical  variables,  and  to  power  manage 
to  delatch  susceptible  circuits.  Each  of 
these  requirements  imposes  additional  sys¬ 
tem  overhead. 

3.1.1  Current  Limiting 

To  provide  adequate  current  limiting,  off- 
chip  resistors  are  required.  For  systems 
employing  SSI/MSI,  this  might  impose  a 
large  overhead  in  power,  weight,  and  vol¬ 
ume.  With  LSI,  however,  this  penalty  is 
proportionally  less  because  fewer  IC's  are 
needed  to  accomplish  the  required  logic 
functions. 

3.1.2  System  Circumvention  and  Recovery 

It  is  standard  practice  to  store  critical 
variables  in  hardened  military  systems  in 
non-volatile  memory  to  prevent  data  loss 
during  radiation  events.  Plated  wire  has 
been  the  preferred  memory  technology,,  but 
the  high  cost  of  this  approach,  and  archi¬ 
tectural  difficulties  which  discourage  dis¬ 
tributed  memories,  m'ake  it  probable  that 
this  will  be  replaced  in  the  near  future 
by  nonvolatile  semiconductor  memory 
(MNOS).  If  such  developments  do  occur, 
a  major  milestone  in  system  level  har¬ 
dening  to  transient  radiation  will  have 
been  achieved. 

3.1.3  Latchup 

Radiation  induced  latchup  can  cause  des¬ 
tructive  damage  to  a  chip.  If  external 
current  limiting  prevents  such  damage,  the 
latchup  condition,  at  the  very  least,  pre¬ 
cludes  normal  operation.  To  restore  system 
operation,  chip  power  must  be  interrupted. 


tion  effects  has  proven  to  be  much  more 
difficult  than  was  anticipated  five 
years  ago.  In  view  of  the  problems  en¬ 
countered,  the  overall  approach  to  LSI 
development  for  stragetic  and  military 
space  applications  must  be  re-examined. 
Our  previous  pre-occupation  with  transi¬ 
ent  hardness  restricted  studies  to  DI 
approaches  to  the  exclusion  of  31  ap¬ 
proaches.  Several  31  technologies,  how¬ 
ever,  are  adequately  blu’d  to  neutron 
fluence  and  toted  ionizing  radiation  dose. 
To  take  advantage  of  the  hardness  of 
these  31  technologies,  system  level  ap¬ 
proaches  to  transient  radiation  effects 
must  be  developed. 
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This  is  a  time  of  revolutionary  chanqe 
in  microelectronics.  We  are  in  transi¬ 
tion  from  SSI/MSI  loqic  to  proqrannnable 
subsystems,  from  cataloq  parts  to  custom 
arrays,  from  LSI  to  VLSI/VHSI  complexity, 
and  from  human-dominated  desiqn  and  test 
activities  to  those  same  activities 
dominated  by  computer  support  aids.  It 
is  important  to  tread  the  narrow  line 
between  eschewinq  all  new  technology  on 
the  basis  that  it  is  untried  and  the 
eaqer  pursuit  of  each  new  capability  as 
it  appears.  This  session  is  intended  to 
first  identify  several  of  the  more 
promising  new  technologies  for  space 
missions  so  that  you  can  evaluate  their 
potential  and  then  to  identify  some  of 
the  validation  and  verification  tools 
that  are  used  to  prove-in  these  new 
technologies.  Advanced  space  missions 
are  critically  dependent  on  using  new 
technology  to  the  maximum  possible  ex¬ 
tent,  just  short  of  introducing  relia¬ 
bility  hazards.  Today's  guick  scan  of 
the  technology  menu  should  both  whet 
your  appetite  for  the  new  technologies 
and  provide  some  caveat  emptor  warnings 
against  letting  your  appetites  overwhelm 
your  good  judgment. 
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BUIX  CMOS  FOR  SPACE  SYSTEMS* 

W.  R.  Dawes,  Jr. 

SANDIA  HATIONAL  LABORATORIES 

Introduction 

Microelectronic  systems  for  space  use 
typically  have  component  requirements 
that  are  characterized  by  low  volume, 
high  reliability,  and  severe  environmental 
conditions  such  as  ionizing  radiation. 
Silicon  gate  CMOS  technology  has  applica¬ 
tions  in  space  systems  because  of  its  low 
static  pwer  dissipation,  high  perfor¬ 
mance,  good  noise  margin,  and  relative 
insensitivity  to  threshold  voltage  shifts. 
This  paper  will  review  a  radiation  hard¬ 
ened  bulk  CMOS  silicon  gate  technology 
that  has  demonstrated  megarad  hardness, 
latch-up  immunity,  and  high  reliability. 
This  process  has  been  used  to  fabricate 
and  deliver  high-reliability,  flight- 
quality  parts  of  both  a  microprocessor 
family  and  custom  logic  circuits.  A  CMOS 
radiation -hardened  standard  cell  design 
package  is  being  developed  to  permit  in- 
house  design,  prototype  fabrication, 
design  verification,  and  second  source 
capability  for  government  systems. 

The  nature  of  the  rapidly  expanding  com¬ 
mercial  market  with  its  emphasis  on  high 
volume,  and  by  space  standards,  modest 
environmental  and  reliability  constraints, 
has  increased  the  difficulty  in  obtaining 
qualified  parts  for  many  space  systems. 
Future  successful  circuit  procurement  will 
require  not  only  that  the  device  techno¬ 
logy  is  capable  of  meeting  the  specialized 
space  requirements  but  also  that  guaran¬ 
teed  access  to  the  design  exists  for 
independent  circuit  verification  and  mul¬ 
tiple  source  capability.  This  paper  will 
discuss  a  radiation-hardened  bulk  CMOS 
technology  and  a  design  implementation 
that  has  been  successfully  applied  to 
procure  radiation-hardened  CMOS  LSI  parts 
for  satellite  systems. 

Discussion 

Space  systems  generally  require  circuits 
that  demonstrate  high  reliability,  since 
there  may  be  10-year  system  life  require¬ 
ments,  static  power  dissipation  on  the 
order  of  1  mW  per  LSI  chip,  some  degree 

*This  work  supported  by  the  U.S.  Depart¬ 
ment  of  Energy. 


of  radiation  hardness,  and  finally,  a 
guaranteed  source.  The  radiation  levels 
vary  for  different  systems,  but  for  the 
purposes  of  this  paper,  the  radiation 
environment  is  1  x  10^^  n/cm^,  a  total 
ionizing  dose  of  5  x  10^  rads  (Si)  and 
transient  survivability,  i.e.,  no  latch-up 
at  any  Y  level. 

This  paper  will  discuss  only  the  silicon 
gate  technology,  since  it  offers  appre¬ 
ciable  performance  and  packing  density 
advantages  over  the  metal  gate  technology 
and  moreover  has  become  the  industry 
standard. 

The  radiation  hardening  for  a  bulk  CMOS 
process  is  a  three-step  procedure.  The 
gate  oxide  region  'for  the  active  devices 
is  first  hardened  to  ionizing  radiation 
and  then  the  parasitic  n-channel  field 
oxide  devices  are  hardened  to  prevent 
field  inversion.  This  field  oxide  is 
hardened  either  by  guardbands,  a  closed 
geometry  layout,  field  shields,  or  more 
recently,  a  hardened  field  oxide.  The 
final  hardening  consideration  is  to  pre¬ 
vent  latch-up  which  can  be  initiated  by  a 
transient  ionizing  radiation  exposure. 
MOS  structures  are  inherently  hard  to 
neutrons. 

The  radiation-hardened  process  for  silicon 
gate  circuits  has  been  previously  docu¬ 
mented^  and  basically  utilizes  a  550  X 
gate  oxide  grown  at  1000®C  with  100 
percent  O2  and  minimized  postgate  ther¬ 
mal  cycles.  To  minimize  the  postgate 
thermal  cycles,  the  n+  and  p-*-  regions  are 
formed  by  ion  implantation,  and  a  950°C 
CVD  oxide  is  used  for  the  inter level 
dielectric.  The  step  coverage  of  this 
oxide  satisfies  step  coverage  requirements 
without  the  necessity  of  a  reflow  cycle, 
and  concommitantly  activates  the  ion 
implantation. 

Hardening  the  parasitic  n-channel  field 
oxide  device  is  presently  done  by  layout 
techniques.  For  example,  a  closed  geome¬ 
try  technology  such  as  C^L  surrounds 
all  drains  with  the  gate  electrode,  thus 
there  are  no  parasitic  field  oxide  paths 
between  drains. ^  Other  techniques  such 
as  diffused  guardbands,  which  in  a  silicon 
gate  process  require  an  additional  photo¬ 
delineation  and  diffusion,  or  field 
shields  have  demonstrated  their  effec¬ 
tiveness  at  preventing  field  inversion 
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at  the  slight  expense  (typically  10  per¬ 
cent)  of  packing  density. ^  From  a 
packing  density  viewpoint,  the  optimum 
field  oxide  hardening  approach  from  those 
discussed  is  a  combination  of  diffused 
guardbands  and  field  shields.  A  layout 
technique  utilizing  such  a  combination 
will  be  reviewed  later. 

The  final  hardening  consideration  for  a 
bulk  CMOS  technology  is  the  prevention  of 
latch-up  induced  by  an  ionizing  radiation 
transient  spike  which  can  exceed 
1  X  10^2  rads/sec.  Again,  several  gen¬ 
eric  process  solutions  have  been  developed 
to  prevent  latch-up.  These  include 
minority  carrier  lifetime  control  such  as 
neutron  irradiation  or  gold  doping,  which 
maintains  the  parasitic  npn  and  pnp  beta 
product  to  less  than  unity.^  This 
technique  is  well  established  and  is 
universally  successful  for  CMOS  structures 
utilizing  4  u®  design  rules  with  p-well 
depths  >  5  pm.  For  designs  with  tighter 
design  rules,  the  neutron  exposure 
required  to  prevent  latch-up  exceeds 
1  X  lOl^  n/cm"  due  to  the  narrow 
parasitic  base  widths  and  thus  becomes 
unattractive.  An  alternative  approach, 
which  does  not  require  a  neutron  source 
and  is  applicable  for  tighter  design 
rules,  is  to  use  an  n  on  n-*-  epitaxial 
substrate  idiich  provides  a  shunting  path 
in  the  parasitic  SCR,  thus  preventing 
latch-up. 5 Table  I  illustrates  the 
data  base  for  latch-up  prevention  of  bulk 
CMOS  for  a  variety  of  circuits  and  design 
rules. 

Table  I 

Bulk  CMOS  Latch-Up 
Lifetime  Control 

Sample  size:  >13,000  packaged  parts 

>  6,000  die  (prepackaged) 
Latch-up  Test:  10^-10^®  rads/sec 

(Febetron) 
10^1-10^2  rads/sec 

(Hermes) 

elec trical 

Epitaxial  Substrates 
Sample  size:  12  uafer  lots 

>  300  packaged  parts 
Latch-up  Test:  10^-10^®  rads/sec 

(Febetron) 
lO^^-lO^^  rads/sec 

(Hermes) 

electrical 


Our  first  application  of  the  hardened 
silicon  gate  bulk  CMOS  process  was  to  the 
RCA  designed  1802  microprocessor  family, 
in  particular,  the  1802  microprocessor, 
the  TCC-244  (256  x  4)  RAM,  and  1834  ROM. 
These  circuits  utilize  a  C^L  layout 
which  eliminates  radiation  induced  field 
inversion,  thus  modifying  the  process  to 
radiation  harden  the  gate  oxide  and  to 
prevent  latch-up  would  satisfy  the  radia¬ 
tion  specifications.  The  gate  oxide 
hardening  was  accomplished  by  using  the 
previously  mentioned  oxidation-post  oxi¬ 
dation  sequence  and  a  1  x  10^^  n/cm^ 
radiation  to  control  minority  carrier 
lifetime  and  prevent  latch-up.  The 
radiation  characteristics  of  the  RAM  are 
listed  in  Table  II  for  10  volt  operation. 
The  data  represents  the  range  for  five 
device  lots,  and  all  exposures  were  at  a 
2  X  106  rad/hour  rate  with  the  part  at 
a  10  volt  static  bias. 

Table  II 

TCC-244  RAM  Characteristics 
(10  Volt  Operation) 

DOSE  (rads) 


Idu(®A) 

prerad 

4. 0-5.0 

1  X  105 

4. 0-5.0 

5  X  105 

3. 0-4.0 

1  X  10^ 

2. 0-3.0 

Ij)p(mA) 

2. 5-3.0 

2.4-2. 9 

2. 0-2. 5 

1.0-1. 5 

Taa^^s) 

150-240 

150-240 

250-350 

500-750 

Tyy(ns) 

40-60 

45-65 

70-110 

140-220 

Idd(ua) 

.5-50 

.5-50 

1.0-110 

1.0-110 

Radiation  hardness  represents  only  one 
requirement  for  the  part,  and  reliability 
is  another  major  concern  for  use  of  these 
parts  in  a  space  system.  A  part  qualifi¬ 
cation  procedure  has  been  established  at 
Sandia  for  delivery  of  high-re  1  parts  for 
either  prototype  development  or  to  second 
source  if  necessary.  The  major  features 
of  this  qualification  sequence  are  listed 
in  Table  III. 

The  bum-in  operation  is  a  150°C,  20 

percent  over-voltage  static  bias,  168 
hour  sequence  that  requires  <  5  percent 
part  failure  for  lot  acceptance.  Lots 
exhibiting  a  5  to  15  percent  failure  rate 
at  first  bum-in  are  permitted  a  second 
bum-in,  but  no  third  bum-ins  are  per¬ 
mitted.  This  burn-in  cycle  and  PDA  (per¬ 
cent  defective  allowable)  is  used  to 
assure  that  the  system  can  meet  a  10-year 
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life  requirement  based  on  the  normal 
assumptions  of  failure  distributions  and 
activation  energies.  The  150°C  tem¬ 

perature  is  required  to  meet  the  relia¬ 
bility  specification  without  resorting  to 
excessive  bum-in  times. 

To  date,  over  1200  parts  of  the  1802 

family  have  been  supplied  for  satellite 
systems  demonstrating  that  the  bulk  CMOS 
technology  is  capable  of  meeting  the 

severe  environmental  and  reliability 

constraints  imposed  by  space  systems. 

Table  IH 

Part  Qualification 
Wafer  Acceptance 

Radiation  Qualification 
Ionic  Contamination 
SEM 

100  Percent  Die  Visual 
QC  Verification 

Die  Shear  and  Bond  Pull  Qualification 

100  Percent  Prelid  Visual 

QC  Verification 

Preseal  Bake 

High -Temperature  Storage 

Temperature  Cycle 

Centrifuge 

PIND  Test 

Fine  and  Gross  Leak  Test 
100  Percent  External  Visual 
QC  Verification 
First  Electrical 
Bum-in 

High-Low  Temperature  Test 
QC  Final 


Part  procurement,  however,  requires  not 
only  that  the  technology  be  capable  of 
meeting  system  objectives,  but  also  that 
a  design  capability  exist  to  support  the 
system.  This  is  essential  for  radiation- 
hardened  parts,  since  the  constraints 
imposed  by  radiation  induced  parametric 
shifts  often  have  a  deleterious  effect 
upon  circuit  functionality.  Many  his¬ 
torical  cases  exist  Miere  connercial 
designs  have  been  "hardened"  by  applying 
the  proper  process  cycle  only  to  discover 
that  the  circuit  would  not  function  due 
to  radiation-induced  parametric  shifts. 
The  explosive  demand  in  the  conmercial 
semiconductor  market  is  also  increasing 
the  difficulty  in  obtaining  radiation- 
hardened  designs  for  tdiat  typically  are 
low-volume  space  applications. 


To  satisfy  the  present  and  future  needs 
for  custom  parts  for  space  and  other 
government  systems,  Sandia  has  developed 
a  silicon  gate  radiation-hardened  bulk 
CMOS  cell  family  with  a  corresponding  CAD 
capability.  The  technology,  referred  to 
as  the  ELA  (expanded  linear  array)  cell 
family  features  a  diffused  p^-  guardband 
for  field  isolation  along  the  channel 
width  direction  and  field  shields  for 
isolation  between  adjacent  n-channel 
devices.^  The  standard  cells  are 
double  ported.  Presently,  18  standard 
cells  have  been  designed,  modeled,  and 
characterized.  Table  IV  lists  the  avail¬ 
able  cells  and  their  width.  The  cell 
heights  are  ISO  um- 

Table  IV 

ELA  Standard  Cells 


Cell  Width  (um) 

Asynchronous  flip-flop-reset  300 

Synchronous  flip-flop-reset  280 

Synchronous  flip-flop-set/reset  320 

Multiplexer  140 

Inverting  Buffer  60 

2- input  NOR  60 

3- input  NOR  80 

4- input  NOR  100 

2- input  OR  80 

3- input  OR  100 

4- input  OR  120 

Exclusive  OR  160 

2- input  AND  80 

3- input  AND  100 

4- input  AND  120 

2- input  n  AND  60 

3- input  n  AND  80 

4- input  n  AND  100 


The  ELA  technology  has  been  used  for  the 
design  and  fabrication  of  an  ELA  test 
chip,  ALU,  and  1024  x  1  RAM.  A  1024  x  8 
ROM  has  been  designed  and  is  presently  at 
the  mask  fabrication  cycle.  The  RAM  has 
a  10  volt,  preradiation  100  ns  access 
time.  The  performance  of  a  megarad  hard¬ 
ened  part  is  typically  less  than  a  non- 
hardened  part  due  to  the  increased  device 
threshold  voltages  and  reduced  n-channel 
mobility. 

The  radiation  response  for  several  ELA 
cells  is  illustrated  in  Figure  1.  This 
observed  performance  degradation  for  a 
total  dose  in  excess  of  1  x  10^  rads 
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Figure  1 

(Si)  is  a  result  of  the  substantial 
p-channel  radiation-induced  threshold 
voltage  shift.  Idq  is  typically 
<100  uA  for  a  hardened  bulk  CMOS  LSI  cir¬ 
cuit  under  static  bias  and  remains 
unchanged  as  a  function  of  total  dose  up 
to  exposures  of  1  x  10^  rads  (Si). 

Future  (and  even  some  present)  system 
requirements  will  need  an  increased  per¬ 
formance  and  packing  density  than  that 
which  is  available  with  the  present 
radiation -hardened  bulk  CMOS  technology. 
To  meet  these  new  requirements,  it  is 
anticipated  that  process  innovations  such 
as  refractory  metal  gates,  recessed  hard¬ 
ened  field  oxides,  bipolar  output  drive 
circuits,  and  device  scaling  will  be 
applied  for  a  new  generation  bulk  CMOS 
hardened  technology.  It  is  also  important 
that  these  new  hardened  processes  be  com¬ 
patible  with  commercial  processes  to 
fabricate  the  necessary  technology  trans¬ 
fer  for  volume  procurement. 

Summary 

Microelectronic  circuits  for  space  systems 
generally  have  stringent  requirements  on 
device  reliability,  radiation  hardness, 
and  static  power  dissipation.  The  radia¬ 
tion  hardened  bulk  CMOS  silicon  gate 
technology  has  successfully  met  these 
requirements  in  deliveries  to  several 
satellite  programs.  A  proven  technology, 
however,  is  only  one  aspect  in  obtaining 
the  necessary  circuits  for  special  sys¬ 
tems.  Circuits  must  be  designed  and 
verified,  especially  for  radiation  envi¬ 
ronments,  and  second  source  capability 
must  exist  to  ensure  meeting  schedule 


commitments.  In  many  cases,  this  means 
that  the  user  organization  must  own  access 
to  the  mask  sets.  A  radiation -hardened 
bulk  CMOS  silicon  gate  standard  cell 
design  package  has  been  developed  idiich 
permits  an  in-house  design,  prototype 
fabrication,  design  verification,  and 
second  source  capability.  This  technology- 
design  system  has  been  successfully 
applied  in  processing  high-reliability, 
radiation-hardened  CMOS  parts  for  DOE 
systems. 
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Current  space  applications  impose  tremen¬ 
dous  demands  on  satellite  digital  elec¬ 
tronics.  Reliable,  high  speed  circuits 
are  required  with  stringent  constraints 
on  size,  weight  and  power.  These  demands 
translate  into  high-performance,  high 
density  large  scale  integrated  circuits 
that  must  also  be  immune  to  large  total 
dose  accumulations  of  radiation  encoun¬ 
tered  from  repeated  passes  through  the 
Van  Allen  belt.  In  addition,  military 
applications  often  insist  upon  devices 
which  are  insensitive  to  bursts  of  gamma 
and  neutron  radiation. 

Complementary  metal-oxide-silicon  (CMOS) 
devices  have  high  immunity  to  neutrons 
and  offer  the  additional  advantages  of 
small  size,  light  weight  and  low  power. 
When  CMOS  arrays  are  processed  appropri¬ 
ately  (special  channel  oxide  formation 
and  subsequent  low-temperature  exposures), 
they  can  be  made  hard  to  large  total  doses. 
By  combining  CMOS  circuitry  with  the 
silicon-on-sapphire  (SOS)  technology, 
one  achieves  higher  speed,  greater  pack¬ 
ing  density,  reduced  dynamic  power,  and 
a  dramatic  decrease  in  sensitivity  to 
bursts  of  gamma  radiation.  These  advan¬ 
tages  have  been  derived  from  the  insula¬ 
ting  sapphire  substrate  and  the 
resultant  minimization  of  P-N  junction 
areas . 

RCA  is  under  contract  with  the  Air  Force 
to  develop  an  LSI  radiation-hardened 
CMOS/SOS  general  processor  unit  and 
associated  circuits.  Figure  1  shows 
photomicrographs  of  some  of  the  chips 
in  the  family. 

The  heart  of  this  chip  set  is  a  general 
processor  unit  (Fig.  2),  which  is  an 
8-bit  parallel  slice.  The  chip  is 
slightly  over  200  mils  on  a  side,  has 


about  2900  transistors,  and  has  been  pro¬ 
duced  at  RCA  by  the  standard  and  radiation 
hardened  processes.  Both  types  have  been 
exhaustively  tested  by  Questron  Corpora¬ 
tion  and  found  to  be  functionally  correct 
and  to  perform  wel 1 . 

All  the  members  of  the  GPU  chip  family, 
shown  in  Table  1,  are  self-aligned  silicon- 
gate  CMOS/SOS  circuits.  They  do  not  in¬ 
corporate  the  special  topological  adjust¬ 
ments,  such  as  tie  down  of  silicon  islands, 
that  are  employed  to  achieve  the  highest 
possible  levels  of  total  dose  hardness. 
However,  they  have  been  designed  with 
yield  and  manufacturability  as  prime 
requirements.  They  have  been  produced 
with  the  RCA-developed  radiation-hardened 
CMOS/SOS  process,  and  are  expected  to 
exceed  total -dose  hardness  levels  of 
50,000  rads(Si)  and  transient  hardness 
levels  of  109  rads(Si)/s.  These  goals 
have  been  surpassed  in  the  first  two 
designs  to  be  radiation  tested,  the  GPU 
and  RAM.  All  of  the  array  types  shown, 
with  the  exception  of  the  Emulating  Con¬ 
troller,  have  been  produced  and  proven 
out  using  the  standard  CMOS/SOS  process. 

The  GPU,  RAM,  and  Multiplier  are  hand¬ 
crafted  custom  arrays;  the  two  control¬ 
lers  are  standard-cell  designs;  and  the 
ROM  and  GUA's  are  customized  by  using  a 
single  mask.  Four  GUA  customizations 
have  been  successfully  designed  and  pro¬ 
duced  for  the  Air  Force  and  evaluated  by 
Tracer,  Inc.  This  chip  set  is  being  con¬ 
sidered  for  several  military  systems. 

A  photomicrograph  of  the  TCS  150,  a  256x4 
bit  RAM,  is  shown  in  Fig.  3.  While  this 
chip  is  not  outstanding  in  number  of  bits- 
per-chip,  it  has  performed  well  under 
radiation. 

Radiation  data  on  several  other  RCA  CMOS/ 
SOS  RAMs  is  presented  in  Table  2.  These 
RAMs  include  the  commercially  available, 
standard  processed,  CDP  1821  and  MWS5114, 
IK  and  4K  static,  random-access  memories. 
The  TCS  191,  which  is  an  earlier  and 
larger  version  of  the  MWS5114,  has  been 
produced  with  the  newer  N+  gate  rad-hard 
process. 

Figure  4  shows  a  custom  CMOS/SOS  Arithme¬ 
tic  Logic  Unit/Test  Array  that  was  de¬ 
signed  using  the  special  topological 
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adjustment  for  increasing  the  immunity  to 
accumulations  of  gamma  radiation.  These 
topological  adjustments  include  the  tie¬ 
down  of  certain  transistor  sil icon  islands 
to  a  fixed  potential  and  limiting  the 
series  stacking  of  transistors  to  three. 
There  was  an  area  penalty  of  about  15% 
for  these  topology  adjustments. 

The  processing  of  CMOS/SOS  arrays  for  en¬ 
hanced  tolerance, to  accumulated  gamma  radi¬ 
ation  involves  moderate  changes  from  the 
standard  silicon-gate  CMOS/SOS  process.  The 
main  differences  are  in  the  pyrogenic  chan¬ 
nel  oxide  growth,  the  avoidance  of  subse¬ 
quent  exposures  to  high  temperatures,  the 
formation  of  the  doped-sil icon  gates,  and 
the  depositions  of  the  field  oxide  and  the 
aluminum.  Radiation-hardened  CMOS/SOS  ar¬ 
rays  have  been  produced  using  either  P+  or 
N+  doped  silicon  gates.  The  N+  doping  is 
preferred  because  of  its  higher  conductiv¬ 
ity. 

Table  3  summarizes  the  radiation  tests 
for  the  various  array  types.  In  the 
transient  tests  a  10-Mev  Linac  electron 
beam  with  both  short  and  long  pulses  was 
used  to  determine  the  upset  levels  and 
transient  annealing  properties  of  the 
radiation-hardened  TCS  129  GPU  and  the 
TCS  150  RAM.  The  GPU  was  exposed  with 
both  a  dominant  "0“  and  dominant  "1" 
pattern  stored  in  its  16  registers. 
Similarly,  the  RAM  was  exposed  with 
dominant  "I's"  and  "O's"  stored  in  the 
four  memory  quadrants. 

On  the  GPU  the  "carry  out",  the  "all 
zero  out",  and  two  of  the  eight  data 
outputs  were  monitored.  On  the  RAM 
all  four  data-output  lines  were  monitored. 
During  the  various  operational  modes  a 
search  was  made  for  the  worst-case  upset 
condition.  Observations  were  made  during 
the  low,  high,  high-impedance,  and 
transition  states.  The  other  array  types 
were  tested  only  for  sensitivity  to  total 
dose  damage  effects. 

The  four  output  waveforms  in  Fig.  5  are 
of  the  GPU  in  a  normal  mode  without  the 
presence  of  radiation.  The  D0(0)  and 
D0(1)  lines  in  Fig.  5a  and  5b  were  pre¬ 
sented  on  a  digitizer  scope.  The  out¬ 
put  signals  are  shown  varying  from  a 
high  to  a  low  state  and  then  to  the  final 
high  impedance  state.  The  large  spike 
in  Fig.  5a  is  a  normal  transient  gener¬ 


ated  within  the  device  as  the  control 
logic  is  changed  from  an  ADD  to  an  AND 
function.  A  smaller  spike  in  the  D0(1) 
output  is  evident  in  Fig.  5b.  The  All- 
Zero-Output  and  the  Carry-Out  signals 
on  dual-beam  oscilloscopes  are  shown  in 
Fig.  5c  and  5d.  A  pin  diode  was  used 
to  normalize  the  dose  rate.  The  supply- 
current  surge  is  shown  above  the  Carry- 
Out  signal.  Since  the  radiation  pulse 
could  be  specified  to  occur  anywhere  in 
these  waveforms,  the  worst-case  condi¬ 
tions  for  upset  to  occur  were  determined. 

Some  typical  upset  responses  are  shown 
in  Fig.  6.  Temporary  upset  is  the 
occurrence  of  a  transient  signal  on  any 
output  line  having  an  amplitude  greater 
than  30%  of  the  Vdd  supply  voltage  and 
disappearing  after  the  radiation  pulse. 
Permanent  upset  is  the  change  in  state 
of  stored  data  which  can  be  corrected 
only  by  refreshing  the  original  stored 
pattern  in  the  registers.  A  comparison 
of  the  two  data  output  line  waveforms 
with  those  in  Fig.  5  reveals  that  a 
change  in  state  occurred  during  exposure 
to  a  dose  rate  of  1.4x1010  rads(Si)/s. 

An  example  of  temporary  upset  occurring 
on  the  AZO  and  on  the  Carry-Out  lines 
is  shown  in  Fig.  6c  and  6d.  This  example 
was  the  worst  case,  with  only  small  tran¬ 
sient  signals  occurring  on  the  data  out¬ 
put  lines.  The  amplitude  of  the  tran¬ 
sient  responses  to  the  radiation  pulse 
as  shown  on  the  AZO  and  CO  lines  is 
about  8  volts,  the  full  Vdd  supply 
voltage.  The  output  recovers  to  its 
original  state  and  is  described  as  a 
temporary  upset. 

Table  4  lists  the  transient  radiation 
results  obtained  with  a  40-ns  Linac  pulse. 
Dose  rates  for  both  temporary  and  perma¬ 
nent  upset  are  shown  for  the  GPU  and  the 
TCS  150  RAM.  The  dose  rates  obtained 
with  both  stored  patterns  in  the  GPU  are 
shown,  the  worst  case  being  the  "0" 
pattern.  Although  the  upset  levels  were 
not  determined  precisely,  they  were 
at  least  a  factor  of  two  higher  than 
those  for  the  "0"  pattern.  All  of  these 
upset  dose  rates  were  measured  for  the 
worst-case  operating  conditions,  which 
were  established  by  varying  the  occur¬ 
rence  of  the  radiation  pulse  relative 
to  the  operation  of  the  device.  It  was 
found  that  both  temporary  and  permanent 


average  upset  rates  are  above  10^0  rads 
(Si  )/s. 

One  of  the  objectives  during  this  series 
of  tests  was  to  characterize  the  GPU 
transient  annealing  response  —  the  abil¬ 
ity  to  recover  from  a  large  dose  per 
pulse  and  to  resume  operational  status. 
Figure  7  shows  the  effect  of  a  27-kilorad 
pulse  on  a  data  output  line.  The  subse¬ 
quent  shutdown  of  device  operation  was 
about  2  us,  with  87X  recovery  of  voltage 
amplitude  within  20  us.  The  system  re¬ 
quirements  determine  what  the  allowable 
recovery  time  may  be.  It  was  observed 
that  the  recovery  time  was  a  function  of 
the  dose  per  pulse  and  previous  dose 
history. 

One  of  the  by-products  of  the  transient 
tests  was  the  measurement  of  total  dose 
characteristics.  These  results,  together 
with  Co^O  test  results,  are  shown  in 
Table  5.  These  arrays  were  produced  with 
the  p+  gate,  radiation-hardened  process. 
Failure  did  not  occur  after  exposure 
for  any  dose  for  all  three  array  types. 

The  determination  of  functionality  of 
a  device  under  test  was  the  same  for  the 
Co60  tests  as  the  pulse  tests.  However, 
in  ..e  case  of  the  TCS  125  array,  a  more 
comprehensive  functional  test,  using  the 
Datatron  tester,  was  employed.  These 
results  clearly  indicate  a  capability 
of  105  to  106  rads  (Si)  for  these  three 
array  types. 

Table  6  lists  the  Co^O  test  results  for 
two  memories  fabricated  with  the  standard, 
commercial,  unhardened  n+  gate  process, 
and  for  the  TCS  191  memory  which  was 
produced  with  an  n+  gate,  radiation- 
hardened  process.  The  TCS  191  is  elec¬ 
trically  identical  to  the  MWS5114  memory. 
The  pass  and  fail  doses  for  all  samples 
are  shown.  On  average  the  commercial 
memories  failed  at  about  10  kilorads, 
while  the  radiation -hardened  memory 
failed  at  an  order-of-magni tude  higher. 

Cosmic  rays  which  exist  in  space  can 
induce  soft  errors  in  solid-state 
memory  devices.  These  effects  can 
occur  in  bipolar,  NMOS,  PMOS,  and  bulk 
CMOS  or  CMOS/SOS  memories,  including 
both  static  and  dynamic  types.  A  limited 
number  of  bulk  CMOS  types  have  been  shown 
to  latchup  in  a  simulated  cosmic  ray 


environment.  Table  7  lists  some  results 
of  cosmic  ray  simulation  experiments  and 
analysis  carried  out  with  CMOS/SOS  and 
bulk  CMOS  memories.  The  first  three  de¬ 
vice  types  are  RCA  CMOS/SOS  memories, 
whereas  the  fourth  is  a  bulk  type.  The 
predicted  error  rates  for  these  memories 
have  been  determined.  Calculation  of 
these  rates  is  based  on  the  experimental 
results  and  on  an  analysis  which  takes 
into  account  the  sensitive  volume  of  the 
memory  storage  cells,  together  with  an 
assumed  cosmic  ray  environment.  The 
table  lists  the  errors  per  day  of  each 
device  for  a  space  craft  containing 
a  memory  system  of  2.5x10^  bits.  The 
Mean  Time  Before  Error  is  listed  for  each 
type.  The  SOS  memories  are  about  one  to 
two  orders  of  magnitude  less  sensitive 
than  the  bulk  device  and  are  immune  to 
latchup. 

SUMMARY 

1 .  The  test  data  have  demonstrated  that 
transient  upset  levels  for  the  complex 
microprocessors  and  memories  are  lOlO 
rads  (Si)/s,  and  that  total -dose  capa¬ 
bilities  are  in  the  range  of  105  to 
10^  rads  (Si ) . 

2.  The  combination  of  special  topological 
adjustment  with  radiation-hardened 
CMOS/SOS  processing  can  guarantee  a 
total -dose  level  of  105  rads  (Si), 
with  sufficient  margin  to  reduce  the 
need  for  a  hardness-assurance  program. 

3.  The  high  degree  of  immunity  from  soft 
errors  induced  by  cosmic  rays  or  alpha 
particles  in  CMOS/SOS  memories,  as 
well  as  the  complete  immunity  from 
latchup,  makes  CMOS/SOS  devices  very 
attractive  for  space  application. 

4.  Radiation-hardened  CMOS/SOS  LSI  arrays 
should  be  seriously  considered  for  use 
in  future  spacecraft  applications. 


294 


III! 


ADVANCES  IN  GALLIUM  ARSENIDE 
INTEGRATED  CIRCUIT  TECHNOLOGY 

A.  Firstenberg,  R.  C,  Eden,  S.  I.  Long, 

R.  Zucca  and  B.  M.  Welch 

Rockwell  International/ 

Electronics  Research  Center 

Thousand  Oaks,  CA  91360 

The  principal  requirements  of  a  digital 
integrated  circuit  technology  compatible 
with  ultra-high  speed  large  scale  and 
very  large  scale  integration  are:  (1)  low 
chip  area  per  gate,  (2)  low  gate  power 
dissipation,  (3)  extremely  low  dynamic 
switching  energy,  (4)  high  speed  and 
(5)  high  process  yield.  The  origins  of 
these  requirements  are  obvious.  Large 
numbers  of  gates,  in  the  10^  to  10^  range, 
cannot  be  placed  on  a  reasonable  sized, 
approximately  1  square  centimeter  chip, 
unless  the  gate  areas  are  small 
(<~  lOOOym^).  The  power  per  gate  must 
be  well  belowl  mW  if  chip  dissipations 
are  to  remain  manageable.  The  constraint 
on  dynamic  switching  energy  for  high 
speed  VLSI  is  especially  severe  because 
chip  power  dissipation  must  be  maintained 
at  high  clocking  frequencies.  For 
example,  assuming  a  maximum  dissipation 
per  104  gate  chip  of  2  watts  at  an 
average  gate  clocking  frequency  of  1  GHz, 
the  maximum  allowing  gate  dynamic  switch¬ 
ing  energy  (Pg'^g)  is  0.1  pJ.  It  is  the 
purpose  of  this  paper  to  report  recent 
advancements  in  Rockwell's  ion  implanted, 
planar,  GaAs  integrated  circuit  technology 
and  to  demonstrate  compliance  between 
this  technology  and  LSI/VLSI  requirements. 

The  performance  advantages  offered  by 
GaAs  integrated  circuits  in  the  areas  of 
speed  and  power  4issipation  places  it  as 
a  prime  candidate  for  the  device  tech¬ 
nology  of  next  generation  military 
electronic  systems.  Advantages  in  both 
speed  and  power  are  derived  from  the 
superior  material  characteristics  of  GaAs 
when  compared  to  silicon.  The  electron 
mobility  in  GaAs  is  approximately  5  times 
that  in  correspondingly  doped  silicon, 
and  approximately  10  times  the  mobility 
of  silicon  grown  on  sapphire.  The 
fabrication  of  GaAs  integrated  circuits 
in  a  semi-insulating  substrate  results 
in  a  reduction  of  stray  capacitances 
leading  to  a  significant  improvement  in 
power  dissipation.  These  two  factors; 
the  high  electron  mobility  of  GaAs  and 


the  availability  of  semi -insulating 
substrate  material  form  the  basis  from 
which  GaAs  integrated  circuits,  operating 
at  significantly  higher  speeds  and  with 
much  lower  dynamic  switching  energy,  can 
be  derived. 

Technology  goals  addressing  LSI/VLSI 
circuits  have  provided  the  motivation  for 
choosing  a  high  yield  planar  fabrication 
approach  and  a  low  power  Schottky  diode 
FET  logic  (SDFL)  circuit  approach.  The 
fabrication  process  capitalizes  on  the 
proven  uniformity,  reproducibility  and 
low  cost  of  implanting  directly  into 
semi-insulatinq  GaAs  for  the  formation 
of  active  device  layers.  The  planar 
circuits  are  fabricated  by  using  multi¬ 
ple  localized  ion  implants  permitting 
the  active  layers  of  Schottky  diodes 
and  FETs  to  be  individually  optimized. 

The  planar  MESFET  is  fabricated  with  two 
implants:  one  shallow,  lightly  doped  n“ 
impla-nt  forming  the  channel  region,  and 
the  other  deeper  n+  implant  forming  the 
source  and  drain  regions  as  shown  in 
Fig.  1.  In  this  manner,  self  aligned 
gates  (Lg=  1  pm)  requiring  =  0.75  pm 
alignment  accuracy  are  formed  having 
excellent  device  characteristics.  The 
dielectric  regions  are  utilized  for  post 
implantation  annealing,  protecting  the 
GaAs  surfaces  during  processing,  and 
passivating  the  surface. 

Inasmuch  as  GaAs  Schottky-barrier  diodes 
are  among  the  fastest  switching  semi¬ 
conductor  devices  existent  and  their 
switching  energies  ('lO’^Sj)  and  required 
areas  (1  pm  x  2pm  active  area)  are  very 
low,  their  use  as  logic  elements  in  GaAs 
digital  ICs  is  very  desirable.  We  have 
developed  a  logic  circuit  approach  using 
high  speed  switching  diodes  as  the  pri¬ 
mary  nonlinear  logic  element,  with  GaAs 
Schottky  gate  FETs  used  for  inversion 
and  gain  as  shown  in  Figure  2.  This 
Schottky  diode  FET  logic  (SDFL)  approach 
results  in  major  saving  of  chip  area 
compared  to  previous  approaches  in  which 
FETs  have  been  used  as  logic  elements. 

Great  strides  have  been  taken  toward  the 
successful  development  of  GaAs  integrated 
circuit  technology.  Improvements  in 
processing  techniques  acquired  from  a 
better  understanding  of  device  physics 
as  well  as  empirical  experiments  have 
provided  processing  results  with  better 


uniformities  and  reproducibility  than 
previously  obtained.  It  is  through 
these  processing  improvements  that 
major  accomplishments  have  been  realized 
in  demonstrated  circuit  performance. 

GaAs  integrated  circuit  propagation 
delays  have  been  measured  as  low  as 
Td  =  62  ps  for  Lj.  =  1  urn  NOR  gate  ring 
oscillators.  SDFL  gates  of  only  SOOurn^ 
have  been  fabricated  and  successfully 
operated.  Power  levels  as  low  as  120uW/ 
gate  have  been  demonstrated,  with  dyna¬ 
mic  switching  energies  as  low  as 
Pp^d  “  27  fj  for  Lg  =  1  urn  NOR  ring 
oscillators,  PqTci  =  16  fJ  for  Lq=  1  urn 
inverters  have  been  measured,  and 
circuit  complexities  in  excess  of  300 
gates  have  been  demonstrated. 


Another  ramification  of  the  improvements 
in  processing  has  been  the  ability  to 
develop  circuits  of  increasing  complexity 
Several  circuits  of  MSI  complexity  have 
been  fabricated  and  demonstrated  to  be 
100%  operational,  as  shown  in  Table  1. 

At  the  time  of  this  writing,  fabrication 
and  testing  of  a  5  bit  x  5  bit  parallel 
multiplier  (260  gates)  and  a  2  x  32 
bit  serial/recirculate  serial  shift 
register  (550  gates)  are  in  progress. 
Furthermore,  an  8  bit  x  8  bit  parallel 
multiplier  of  approximately  1000  gates 
in  complexity  is  expected  to  be  demon¬ 
strated  by  July,  1980. 

A  review  of  electronic  requirements  for 
a  variety  of  missions  including  air 
defense,  ocean  surveillance,  theater 
surveillance,  ballistic  missile  defense, 
ASAT  tracking  and  imaging,  ESM,  intelli¬ 
gence,  and  military  satellite  coimiuni- 
cations,  clearly  dictates  the  require¬ 
ment  for  VLSI  complexity  (il0,000  gates). 
Within  these  missions,  specialized 
circuits  are  required  for  a  multitude 
of  signal  processing  and  communication 
tasks  and  VLSI  computational  and  memory 
circuits  common  to  many  mission  applica¬ 
tions  are  needed.  Extension  of  the  GaAs 
technology  into  the  VLSI  range  would  make 
possible  the  fabrication  of  ultra  high 
speed,  low  power  integrated  subsystems 
which  cannot  be  produced  using  any  other 
present  IC  technology. 


The  Impact  to  system's  considerations  in 
both  Increased  capabilities  and  simpli¬ 
fied  system  architectures  is  enormous. 
Algorithms  of  substantially  increased 


sophistication  could  be  performed. in  the 
same  length  of  time,  or  solutions  to 
processing  tasks  could  be  available 
within  greatly  reduced  times.  In  terms 
of  guidance  and  control  systems,  more 
elaborate  processing  algorithms  trans¬ 
lates  to  improved  performance  particu¬ 
larly  in  low  contrast  (night  time)  and 
smoke  countermeasure  engagements.  Sig¬ 
nificant  improvements  in  countermeasure 
immunity  may  be  realized  by  permitting 
calibration,  filtering,  correlation, 
computation,  computational  verification 
and  correction  all  within  one  control 
frame  period.  Computational  availability 
in  shorter  periods  of  time  reduces 
guidance  error  amplitudes  and  can  be 
traded  off  with  other  subsystem  speci¬ 
fications  improving  manufacturability  and 
cost  effectiveness  of  elements  outside 
the  electronic  subsystem. 

Data  acquisition  and  storage  processors 
common  to  surveillance  missions  can  be 
greatly  simplified  by  the  existence  of 
high  speed  memory.  Multiplexing  and 
demultiplexing  circuitry  required  to 
achieve  compatibility  with  lower  speed 
memory  could  be  greatly  reduced.  Further¬ 
more,  a  high  speed  GaAs  information 
preserving  data  compressor  and  subsequent 
decompressor  could  be  utilized  to  reduce 
system  memory  requirements  by  a  factor 
of  between  2  and  4  depending  on  the 
application.  When  one  considers  that 
approximately  18,000  16K  bit  RAMs  are 
required  for  certain  applications,  the 
savings  of  9,000  devices  per  processor 
has  a  significant  impact  on  both  system 
reliability  and  system  cost.  It  is 
important  to  note  that  memory,  not 
computation  generally  determines  processor 
size,  weight  and  power. 

In  computational  systems,  complex  multiply- 
add  operations  often  dominate  timing 
considerations.  In  both  air  defense  and 
ocean  surveillance  missions,  computation¬ 
al  throughput  is  determined  primarily 
by  the  rate  which  Fast  Fourier  Transforms 
(FFT)  can  be  performed.  High  speed 
operation  can  be  traded  off  in  a  number 
of  ways  to  improve  system  functional 
capabilities.  The  achievement  of  very 
complex  ultra  high  speed  GaAs  VLSI 
circuits  would  define  new  dimensions  in 
on-chip  computational  power.  A  10^  gate 
GaAs  chip  with  =  loO  ps  operating  in 
a  pipeline  processor  would  have,  assuming 
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an  fp  =  1/5  clock  frequency,  a  gate 
count-clock  rate  product  of  2  x  10l4  per 
second  -  a  prodigious  computation  rate. 
The  achievement  of  this  level  of  GHz 
rate  computational  power  in  a  low  power 
integrated  circuit  could  make  possible 
systems  approaches  in  high-speed  signal 
processing,  far  beyond  present  electronic 
system's  capabilities. 
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IMPACT  OF  VHSIC  TECHNOLOGY 
ON  AIR  FORCE  SYSTEMS 

John  M.  Blaslngame 

AIR  FORCE  WRIGHT  AERONAUTICAL  LABORATORIES 

The  Very  High  Speed  Integrated  Circuits 
(VHSIC)  program  Is  a  large  ($200M) ,  trl- 
servlce,  new  Initiative  to  capitalize  on 
new  technological  opportunities  In  very 
large  scale  Integrated  circuit  technology 
to  produce  advanced  weapon  systems  In  the 
1985-1990  time  period.  The  program  Is  or¬ 
ganized  as  Illustrated  In  Figure  1,  show¬ 
ing  the  VHSIC  Program  Schedule,  divided 
Into  four  distinct  phases.  Phase  0  con¬ 
sists  of  a  9  months  Program  Definition 
study  to  analyze  the  various  system  can¬ 
didates  and  partition  these  systems  Into 
functional  modules.  Commonality  of  these 
modules  across  as  many  systems  as  possible 
will  be  emphasized.  Specific  design  and 
fabrication  approaches  for  selected  sets 
of  these  common  modules  (or  chips)  will 
be  determined  during  the  study  phase  and 
Incorporated  Into  a  proposal  for  Imple¬ 
mentation  In  Phase  1  of  the  program.  Nine 
contracts  for  the  Phase  0  study  were  a- 
warded  on  7  March  1980  as  Indicated. 

Phase  1  will  narrow  down  to  approximately 
3-6  contracts  to  continue  a  three  year 
development  of  1.25  micrometer  minimum 
feature  chips  as  well  as  the  development 
of  a  pilot  line  to  produce  these  chips. 

In  addition  the  feasibility  of  0.5-0. 7 
micrometer  minimum  feature  chips  will  be 
determined  during  Phase  1. 

Phase  2  Is  a  two  year  effort  to  take  the 
1.25  micrometer  minimum  feature  chips  pro¬ 
duced  In  Phase  1  and  Incorporate  these  In 
electronic  brassboards  In  order  to  demon¬ 
strate  their  operation  In  selected  systems. 
In  addition  a  pilot  line  will  be  develop¬ 
ed  for  the  production  of  chips  with  0,5- 
0.7  micrometer  minimum  features. 

Phase  3  Is  a  concurrent  effort  to  concen¬ 
trate  a  number  of  relatively  small  con¬ 
tracts  on  special  problem  areas  which  pro¬ 
vide  support  technology  to  the  main  pro¬ 
gram. 

The  overall  objectives  for  the  VHSIC  pro¬ 
gram  as  a  whole  are  given  In  Figure  2. 

The  system  candidates  which  will  be  ana¬ 


lyzed  In  Phase  0  are  listed  in  Figure  3. 
There  are  a  number  of  similar  systems  with¬ 
in  the  three  services,  le:  each  service 
has  a  communication  system,  an  Image  proc¬ 
essor  system,  plus  an  electromagnetic  war¬ 
fare  system;  several  services  have  a  tac¬ 
tical  as  well  as  a  surveillance  radar  sig¬ 
nal  processor,  missile  guidance  system  and 
general  purpose  computers.  All  20  systems 
will  be  considered  during  the  course  of 
Phase  0. 

The  specific  technologies  being  investi¬ 
gated  are  listed  In  Figure  4.  Many  compa¬ 
nies  chose  to  Investigate  several  technol¬ 
ogies,  since  It  Is  not  obvious  at  this 
point  that  any  specific  one  will  meet  all 
the  goals  of  the  VHSIC  program. 

Figure  5  Illustrates  the  overall  goal  of 
the  program -  to  reduce  a  signal  proc¬ 

essor  which  might  take  a  box  of  7  each 
6X9  Inch  circuit  boards  and  reduce  this 
to  just  one  chip  with  VHSIC  technology. 

Figure  6  Illustrates  some  of  the  key  tech¬ 
nology  that  Is  required  to  achieve  the  rev¬ 
olutionary  capabilities  of  VHSIC  chips. 
Certainly  one  of  the  most  Important  will 

be  the  use  of  electron  beam  lithography - 

either  by  focused  electron  beams  drawing 
directly  on  the  surface  of  a  resist  coat¬ 
ed  silicon  wafer  or  by  generating  a  master 
mask  of  high  resolution  (ie:0.5  micrometer 
minimum  feature  size  with  +  0. 1  micrometer 
resolution)  which  can  be  printed  on  the 
silicon  wafer  by  a  flood  beam  of  X-rays. 

The  scanning  electron-beam  micro-fabrica¬ 
tor  type  of  machine  possesses  enormous 
potential  for  application  in  the  VHSIC  pro¬ 
gram  because  It  can  be  programmed  directly 
by  a  computer.  Indeed,  the  application  of 
an  entire  hierarchy  of  computer  aided  sys¬ 
tem  analysis,  through  computer  aided  de¬ 
sign,  computer  aided  lithography,  computer 
aided  manufacturing,  and  computer  aided 
testing  Is  crucial  to  the  success  of  the 
entire  VHSIC  program. 

Figure  7  illustrates  the  status  of  Inte¬ 
grated  circuit  technology  In  1980  at  about 
the  16  bit  microprocessor  or  65K  RAM  memory 
level  with  the  3-5  micrometer  optical  lith¬ 
ographic  technology  available  today.  As 
the  VHSIC  program  progresses  toward  the 
goal  of  0.5  micrometer  minimum  feature  size 
devices,  we  should  see  32  bit  microproc¬ 
essors  with  1MB  memories  In  the  late  1980s. 
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Figure  8  shows  the  ultimate  physical  lim¬ 
its  for  how  far  microelectronic  technol¬ 
ogy  can  push  Integrated  circuits  in  terms 
of  speed  and  power,  if  limited  in  power 
dissipation  to  about  2  watts  per  centi¬ 
meter  square  chip.  The  speed  of  light  a- 
cross  the  chip  represents  one  ultimate 
barrier  which  cannot  be  exceeded.  The 
thermal  noise  generated  at  any  given  tem¬ 
perature  (le:  room  temperature  shown)  rep¬ 
resents  another  fundamental  barrier.  In 
addition  the  quantum  mechanical  limit 
given  by  Information  theory  serves  as  an 
ultimate  barrj^r.  The  goal  of  the  VHSIC 
program  of  10  gate-clock  cycles  is  shown 
in  relation  to  some  of  the  best  device 
performance  reported  to  date  for  silicon 
and  gallium  arsenide  depletion  mode  MESFET 
discrete  devices  with  1.0  micrometer  gate 
structures.  One  can  observe  from  this  fig¬ 
ure  that  there  is  considerable  room  for 
Improvements  in  Integrated  circuits  from 
a  fundamental  point  of  view,  but  that  one 
is  rapidly  limited  by  the  lithographic 
technology  when  forming  more  gates  per 
unit  area  on  a  chip. 

An  analysis  of  current  radar  systems  and 
their  processor  requirements  has  been  per¬ 
formed  by  Hines  and  Blck  and  was  reported 
at  the  November  1979  West  Coast  Computer 
Conference.  The  following  Illustrations 
are  taken  from  their  analysis.  Figure  9 
shows  the  typical  cost  distribution  in  a 
large  airborne  radar  system.  Note  that  the 
largest  component  cost  is  in  the  signal 
processor  (about  26%).  Thus,  this  compo¬ 
nent  rapidly  becomes  the  limiting  item 
for  Improvement  of  radar  systems.  Figure 
10  Illustrates  the  parameters  for  a  typi¬ 
cal  flying  aircraft  radar  processor  of  to¬ 
day.  Future  tactical  fighter  planes  may 
require  radars  which  operate  in  additional 
modes.  Figure  11  shows  the  required  ma¬ 
chine  parameters  for  a  radar  processor  for 
the  four  tactical  modes  of  (l)alr/alr 
search  and  track,  (2)  IFF,  (3)  alr/ground 
SAR/  GMTI,  and  (4)  terrain  following/ter- 
rain  avoidance.  Using  state-of-the-art 
LSI/ECL,  radar  processors  have  been  con¬ 
structed  with  characteristics  as  shown  in 
Figure  12.  This  represents  an  extremely 
expensive  processor  which  is  generally 
unacceptable  for  a  tactical  fighter. 

The  capabilities  of  a  processor  construct¬ 
ed  with  1.25  micrometer  minimum  feature 
VHSIC  chips,  such  as  should  become  avail¬ 
able  in  1984,  are  shown  in  Figure  13. 


Future  requirements  for  tactical  fighters 
including  reconnaissance  capability  with 
synthetic  aperature  techniques  are  shown 
in  Figure  14.  The  capabilities  achievable 
by  a  radar  processor  made  from  0.5  micro¬ 
meter  minimum  feature  VHSIC  chips,  such  as 
should  be  available  in  1986,  are  shown  in 
Figure  15.  Thus  by  the  late  80 's  VHSIC 
technology  should  approach  the  capability 
of  meeting  the  Air  Force  1980  requirements 
for  tactical  fighter  with  reconnaissance 
capability  as  shown  in  Figure  16.  Without 
VHSIC  this  capability  would  be  totally 
impossible. 

If  we  look  at  the  radar  proces9>r  situation 
another  way,  we  can  reduce  the  Integrated 
circuit  count  and  Improve  the  reliability 
and  performance  of  today's  existing  radars, 
as  shown  in  Figure  17.  Similar  results 
have  been  predicted  from  the  analysis  of 
other  types  of  Air  Force  systems  across 
the  board. 

In  summary.  Figure  18  shows  some  recom¬ 
mendations  which  follow  from  the  VHSIC 
program,  which  likewise  should  apply 
to  all  Air  Force  electronic  systems. 
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AN  INTEGRATED  DESIGN  AND  MANUFACTURING 
APPROACH  TO  VLSI 

Dr.  R.  F.  Schauer 

IBM  Data  Systems  Division,  East  Fishkill 
Hopewell  Junction,  N.Y.  12533 


In  the  early  1970's,  IBM  recognized  the 
design  and  manufactur ing  problems  assoc¬ 
iated  with-  the  exploitation  of  VLSI  for 
main  frame  computer  microelectronics.  The 
IBM  design  philosophy,  which  encourages 
end  product  designers  to  personalize  the 
logic  in  a  given  design  to  uniquely  satisfy 
the  specifications  of  that  end  product,  was 
not  to  be  changed.  Therefore,  projects 
were  put  in  place  to  develop  a  design 
capability  that  would  permit  the  design  of 
LSI  components  (and  extendable  to  VLSI 
components)  anywhere  in  the  corporation 
consistent  with  technology  design  con¬ 
straints,  manufacturing  constraints  and 
quality  constraints. 

Two  design  approaches  were  available.  The 
first,  a  custom  design,  supports  a  unique 
design  tuned  to  the  product  specifications 
while  maximizing  silicon  usage.  Both  the 
design  time  and  the  qualification  time  of 
custom  designs  are  measured  in  months. 

Design  support  normally  consists  of  inter¬ 
active  design  aids.  Because  of  the  cost 
and  .-length  of  time  required  for  design 
and  qualification  of  custom  chips,  it  was 
decided  to  restrict  this  approach  to  a  few 
special  high-volume  part  numbers. 

An  automated  design  approach  was  selected 
as  the  "work  horse"  for  LSI  chip  design. 

A  "mastersl ice"  or  gate  array  chip  design 
is  coupled  with  a  turnkey  design  system  to 
minimize  the  manual  design  effort  while 
assuring  the  manufacturability  and  quality 
of  the  custom  personalized  chips.  This 
approach  minimizes  design  and  qualification 
cost  and  time  at  the  expense  of  design  con¬ 
straints  to  achieve  automation. 

Manufacturing  initially  had  a  number  of 
concerns  about  such  an  automated  design- 
release  system.  First,  it  needed  the 
capability  to  determine  the  manufactura¬ 
bility  of  each  part  number  at  the  time  of 
release  of  that  part  number;  next,  it  needed 
the  ability  to  guarantee  the  quality  and 
reliability  of  these  parts;  and  lastly, 
because  Its  process  lines  will  evolve  with 
time,  it  needed  the  ability  to  assess  the 


reproducibility  of  these  parts  over  time. 

These  concerns  were  addressed  by  first 
establishing  procedures  for  systems  inte¬ 
gration  and  control,  and  secondly,  a  manu¬ 
facturing  audit  capability.  Systems 
integration  and  control  essentially  put 
engineering  level  control  on  the  parts  -■  s 
well  as  the  whole  of  the  design  and  release 
system.  'The  programs,  the  technology 
design  rules,  the  process  design  rules, 
and  the  design  methodology  are  qualified  by 
an  independent  assurance  organization  be¬ 
fore  a  new  technology  is  made  available  to 
end  product  designers.  Changes  and 
enhancements  are  also  qualified  before  they 
are  released  to  the  users. 

The  audit  function  verifies  the  goodness 
of  the  design  data  by  checking  the  inte¬ 
grity  of  the  design  data,  the  engineering 
control  levels  of  technology  rules,  process 
rules,  and  design  tools  used,  and  the 
design  methodology.  Automatic  qualification 
is  granted  a  part  number  if  the  released 
data  audit  is  successful. 

There  are  five  steps  required  to  design  a 
chip  with  the  design  system.  Although 
constraint  of  designing  for  testability 
impacts  the  initial  logic  design,  the  logic 
entry  function  and  design  verification  are 
free  of  constraint  and  are  essentially 
technology  independent.  Physical  design 
is  constrained  by  the  technology  design 
rules  and  the  mastersl ice  layout.  One  of 
the  prime  considerations  in  mastersl ice 
layout  is  the  wirability  of  the  chip.  A 
prediction  of  the  required  number  of 
wiring  channels--considering  the  average 
number  of  connections  per  circuit,  the 
average  connection  length,  the  number  of 
circuits,  and  the  blockage  per  connection 
for  macros--is  made  early  in  the  design  of 
the  mastersl ice.  A  tradeoff  is  made  be¬ 
tween  the  acceptable  confidence  level  for 
wirability  and  the  amount  of  silicon  area 
dedicated  to  wiring.  This  tradeoff  is 
critical  because  it  has  been  shown  that  one 
or  two  wiring  channels  per  cell  can  make 
the  difference  between  wirability  and  non- 
wi rabi 1 1 ty . 


The  automated  physical  design  subsystem 
uses  the  following  strategy  for  the  phys¬ 
ical  iayout.  First,  automatic  placement 
including  a  crude  iteration  of  global 
wiring  demand,  is  completed.  Technology 
ground  rules  such  as  capacitive  loading 
and  iive  voltage  drops  are  considered  in 
this  phase.  Next,  giobal  wiring  is  com¬ 
pleted  with  iteration  on  placement  if  re¬ 
quired;  and  finaliy,  automatic  point-to- 
point  wiring  is  accomplished.  Additional 
technology  design  rules  are  considered  In 
this  phase.  Complex  logic  chips  can  be 
designed  in  a  matter  of  days  using  this 
approach,  with  all  technology  design  rules 
checked  in  the  process.  A  iog?cal-to- 
physicai  verification  is  also  made. 

In  test  generation  the  design  Is  verified 
to  be  consistent  with  the  design  rules  for 
testability.  DC  stuck  fault  patterns  are 
then  generated.  Delay  test  patterns  can 
also  be  generated  If  required.  Testability 
coverage  is  calculated  during  this  process, 
and  generally  that  coverage  will  be  between 
95-100*. 

The  design  data  Is  then  formatted  In  the 
release  subsystem  for  release  to  manufac¬ 
turing.  In  the  manufacturing  release 
system,  the  audit  function  Is  run,  and 
if  successful,  the  part  number  Is  accepted 
and  granted  qualification.  The  release 
system  also  processes  the  design  data  to 
generate  the  necessary  numerical  control 
instructions  to  drive  process  tools  and 
testers  on  the  production  line. 

The  design-release-manufacturing  system 
complex  has  been  used  to  design  and  re¬ 
lease  thousands  of  custom  personalized 
chip  designs  during  the  past  several  years. 
The  audit  of  the  design,  including  test 
pattern  verification,  can  be  accomplished 
in  minutes  compared  to  days  or  weeks  for 
manual  designs.  The  quality  of  the  design 
data,  as  measured  by  zero  wafer  yield 
attributable  to  bad  design  data,  has  been 
excellent.  Less  than  0.1*  of  released 
parts  have  had  zero  yield  because  of  errors 
in  the  design  data.  VLSI  technologies  will 
be  designed  and  manufactured  by  extending 
this  system. 


Conclusions 

1.  Masters  lice  or  gate  array  chips  can 

be  designed  efficiently  and  effectively 
with  turnkey  design  systems. 

2.  Checking  functions  and  methodology 
controls  can  be  Implemented  so  that 
manufacturability  and  quality  can  be 
assured. 

3.  These  design  systems  are  extendable  to 
VLSI. 

Recommendations 

1.  Gate  arrays  should  be  studied  as  a 
technology  vehicle  to  efficiently  and 
effectively  design  low-volume  micro¬ 
electronic  parts. 

2.  An  integrated  design  and  manufacturing 
sys.tem  should  be  used  to  support  the 
gate  array  chip  technology.  Such  an 
integrated  system  should  be  part  of 

a  total  technology  definition. 
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ABSTRACT 

The  trend  in  semiconductor  technology  is  to 
reduce  cost  per  function  by  providing  more  com¬ 
ponents  on  a  single  chip.  The  increased  number  of 
functions  per  chip  also  increases  the  number  of 
input/output  connections  (I/Os)  per  chip.  Increased 
chip  complexity  and  I/Os  have  several  effects:  an 
increased  requirement  to  pretest  and  screen  before 
interconnecting  multiple  cUp  assemblies  and  a  need 
to  improve  the  performance  and  packaging  density  of 
heretofore  acceptable  dual  in-line  packages  (DIPs). 

Compared  to  DIPs,  the  hermetic  chip  carrier 
(HCC)  has  better  hi^  frequency  characteristcs  and  a 
5:1  improvement  in  packaging  density.  The  HCC  is 
being  standardized  by  JEDEG  in  both  40  and  50  mil 
center-to-center  I/O  spacing  in  fam^es  with  up  to 
156  I/Os.  Tooling  of  both  HCCs  arid  test  burn-in 
sockets  with  84  I/Os  is  available. 

Applications  of  HCCs  at  the  module  level  demon¬ 
strate  the  5:1  packaging  density  improvement. 
Deugn,  assembly,  and  test  methods  are  reviewed  for 
both  single  and  double  sided  assemblies.  Results  of 
environmental  tests  are  given.  Advantages  of  the 
packaging  system  are  shown.  Conclusions  are  derived 
and  recommendations  for  future  activity  given. 

INTRODUCTION 

The  trend  in  semiconductor  technology  of  pro¬ 
viding  more  components  on  a  sin^e  chip  has  effec¬ 
tively  reduced  the  coet  and  improved  the  reliability 
per  function.  These  two  advantages  of  larger  scale 
integration  (LSI)  open  up  new  applications  of  elec¬ 
tronics  and  allow  an  increased  number  of  functions 
per  chip  for  electronic  intensive  applications. 

However,  as  the  number  of  components  per  chip 
increases,  the  number  of  I/Os  increases  and  the 
popular  DIP  and  flatpack  packages  are  too  wasteful 
of  space  and  weight.  The  hybrid  packaging  engineer 
din^tes  standard  packages  by  attaching  many  bare 
chips  to  a  thin-  or  thick-film  substrate  with  leads 
bonded  from  chip  to  conductors  on  the  substrate. 
The  unit  is  then  hermetically  sealed  in  a  larger  pack¬ 
age.  Fewer  connections  are  brought  nut  of  the  hybrid 


than  are  connected  internally.  Packaging  density 
improvement  of  10:1  is  usual  with  hybrid  packaging 
compared  to  sinde  chips  in  standard  packages.  As 
long  as  system  complexity  requires  multiple  chips  to 
perform  a  complete  system  function,  a  requirement 
will  exist  to  interconnect  chips  by  hybrids  in  hi^i 
density  packaging  applications. 

PROBLEMS  WITH  COMPLEX  HYBRIDS 

Although  hybrid  packaging  provides  high  density 
with  attendant  volume  and  wei^t  reduction,  the 
difficulty  of  constructing  and  testing  reliable  hybrids 
has  increased  with  the  complexity  of  the  chips.  It  has 
become  more  difficult  to  achieve  high  yield  because 
of  the  inability  to  adequately  screen  and  test  chips 
and  substrates  before  assembly,  and  because  wire 
bonding  assembly  itself  is  not  always  a  high  yield 
process.  These  problems,  which  reduce  yield  and  raise 
costs,  are  further  compounded  by  the  problem  of 
repair  after  sealing  (opening  the  large  hermetic  seal, 
repairing  the  chip,  and  then  resealing  to  hermetic 
conditions).  Repair  can  only  be  accomplished  by 
skilled  people  in  a  special  hybrid  facility. 

Other  solutions  to  chip  and  wire  hybrids  have 
been  tried  in  an  attempt  to  solve  the  various  prob¬ 
lems.  The  passivated  beam-lead  or  tape  bonded  chip 
has  not  gained  favor  because  of  a  variety  of  technical 
difficulties  which  have  not  yet  been  solved,  plus  two 
additional  problems  —  one  associated  with  mature 
chips,  the  other  with  new  chips.  Mature  chips  of  a 
generic  type  produced  by  different  manufacturers  are 
modified  to  their  own  designs  and  processes  to 
improve  chip  yield,  thus  they  differ  in  detail.  Because 
they  do  not  have  the  same  I/O  pad  placement,  they 
will  not  fit  the  same  substrate  bond  pattern  in  the 
case  of  beam  leaded  chips,  or  the  same  tape  in  the 
case  of  tape  bonded  chips.  In  addition,  adequate 
mechanisms  are  not  available  to  screen  out  defective 
chips  by  bum-in  or  other  means  before  committing  to 
a  multiple  chip  hybrid.  New  chips  with  a  large 
number  of  I/Os  are  generally  not  made  with  beam 
leads  or  tape  bonded.  Rather  they  are  made  with 
standard  pads  for  wire  bonding.  The  new  chips,  with 
great  complexity  and  many  I/Os,  generally  do  not 
have  the  maturity  of  experience  to  provide  hi|d>  yield 
and  must  be  thorou^ly  screened  and  tested  before 
final  assembly.  Three  major  requirements  must  be 
met  for  producing  complex  high  density  hybrids  with 
high  yield: 

1.  100%  screening  and  test  of  semiconductors. 

2.  100%  test  of  substrates. 


3.  yield  final  assembly  mechanism  of  semi¬ 
conductors  to  substrate. 

All  three  conditions  are  met  by  mounting  chips  in 
ceramic  leadless  HCCs  and  assembling  and  inter¬ 
connecting  the  HCCs  on  multiple-layer,  thick-film, 
hybrid  substrates. 

A  typical  48-lead  DIP  is  compared  to  a  48-iead 
HCC  in  Figure  1,  which  directly  illustrates  the 
improvement  in  packaging  density  and  indirectly  the 
improvement  in  circuit  characteristics  provided  by 
the  reduced  inductance  and  capacity  of  leads  from 
the  chip  to  the  package  leads.  Typical  results  of  tests 
at  RCA  indicate  that  digital  clock  rates  are  limited  to 
under  500  MHz  by  the  DIP  whereas  time  delay  reflec- 
tometer  (TDR)  measurements  of  the  HCC  show 
potential  operation  to  4  GHz  with  standard  carriers 
and  above  that  with  specially  designed  carriers. 

Chips  can  be  mounted  and  connected  singly  in  a 
chip  carrier  or,  for  purposes  of  standardizing  the 
types  of  carriers,  multiply  mounted  and  connected  in 
groups  of  three  or  four  in  a  larger  chip  carrier  as 
shown  in  Figure  2.  This  view  shows  a  typical  assem¬ 
bly  of  chips  in  chip  carriers  mounted  and  inter¬ 
connected  on  a  multiple  layer  thick-film  substrate. 
The  chip  carriers  have  not  been  hermetically  sealed 
by  covets  in  order  to  show  the  arrangement.  In  the 
production  assembly  the  chips  would  not  be  visible. 
In  the  case  of  the  single  chip  per  carrier,  chips  are 


mounted  on  the  carrier,  and  all  I/Os  on  the  chip  are 
connected  directly  to  the  inner  leads  of  the  chip 
carrier.  In  the  case  of  the  multiple  chips  per  carrier, 
chips  are  mounted  on  a  7  mil  ceramic  sul^rate  with 
sin^e-layer  gold,  thick-film  patterns  to  provide  an 
intermediate  connection  between  chip  bond  out  and 
package  bond  out  which  reduces  the  wire  lead  length 
and  prevents  wire  crossovers.  Most  chip  I/Os  are 
bonded  out.  A  few  paths  such  as  power  and  ground 
are  common,  as  would  be  in  a  larger  LSI.  In  some  few 
cases  where  the  number  of  outside  leads  is  limited, 
chip  to  chip  connections  are  made  internal  to  the 
package.  Thus,  most  of  the  I/Os  are  available  for 
connection  to  electrical  test. 

FABRICATION  COMPARISON 

Fabrication  of  a  conventional  hybrid  is  a  sequen¬ 
tial  operation;  i.e.,  one  must  have  the  substrates,  then 
all  of  the  chips.  The  assembly  process  commences 
with  die  bonding  and  wire  bonding,  followed  by  sub¬ 
strate  attachment  to  the  package  and  then  the  first 
electrical  test.  Two  basic  problems  arise:  (1)  for  com¬ 
plex  chip  and  wire  hybrids  it  is  virtually  impossible  to 
completely  test  the  substrate  by  automatic  equip¬ 
ment  because  adequate  test  probe  fixtures  cannot  be 
made  to  match  the  small  bonding  pad  dimensions; 
and  (2)  the  first  time  the  active  devices  and  substrate 
are  completely  tested  is  in  the  assembled  unit  at  the 
first  electrical  test.  If  either  the  substrate  or  any  of 
the  devices  is  defective,  a  large  repair  operation 


as  presently  exist  on  standard  IC  testers.  Multiple 
chips  in  chip  carriers  are  easily  tested  by  a  simple 
reprogramming  of  most  standard  IC  testers.  Burn-in 
sockets,  cards,  and  frames  are  available  for  com¬ 
ponents  in  chip  carriers,  as  shown  in  Figure  4. 

A  necessary  consideration  in  fabricating^  ^ge, 
complex,  multilayer  substrates  (with  more  than  500 
interconnections  and  interconnecting  20  or  more 
48-pin  chip  carriers  containing  many  LSI  devices)  is 
that  all  substrates  must  be  tested  for  continuity  and 
isolation  of  all  nets.  The  greater  the  substrate  com¬ 
plexity,  the  greater  the  need  to  perform  electrical 
tests.  This  testing  is  very  difficult  and  expensive  to 
implement  with  conventional  chip  and  wire  substrates 
because  the  small  pad  spacing  requires  a  complex 
custom  probe  setup.  On  the  other  hand,  the  pad 
spacing  for  mounting  chip  carriers  allows  fixturing  in 
pogo-pin  testers,  as  illustrated  in  Figure  5.  The  pins 
are  connected  to  an  automatic  continuity  and  isola¬ 
tion  tester;  continuity  and  isolation  parameters  are 
programmable.  Testing  of  substrates  with  this  kind  of 
tooling  can  generally  be  completed  in  less  than  a 
minute. 


DEVICE  FABRICATION  SUBSTRATE  FABRICATION 


Figure  3.  Simplified  Chip  Carrier  Fabrication  Flow  Chart, 
Showing  Parallel  Operations  for  Devices  and 
Substrates 


Figure  4.  Chip  Carrier  Burn-In  Sockets  on  Standard  Circuit 
Cards  and  Frames 


Figure  5.  Substrate  Continuity  and  Isolation  Test  Fixtures 


ASSEMBLY 

Given  substrates  that  are  100%  tested,  and  tested 
and  bumed-in  active  components  in  the  chip  carriers, 
RCA  has  developed  a  high-yield  assembly  operation. 
A  necessary  precondition  for  successful  assembly  is  a 
solderable  metallization  system  on  the  substrate  and 
chip  carriers,  proper  design  of  the  chip  carrier 
mounting  pads  on  the  substrate,  proper  amount  and 
type  of  solder  and  flux,  and  a  soldering  medium  that 
is  well  controlled  and  adaptable  to  various  sizes  of 
substrates.  Chip  carriers  are  semiautomatically  pre¬ 
tinned  in  60/40  solder.  Solder  paste  is  applied  to  the 
substrate.  Chip  carriers,  edge  clips,  and  capacitor  and 
resistor  chips  (where  required)  are  placed  on  the 
tacky  solder  paste,  and  then  the  entire  unit  is  left  to 
dry.  The  tacky  solder  paste  serves  to  hold  the  chips  in 
place  during  the  subsequent  soldering.  Several  solder¬ 
ing  systems  have  been  used  successfully,  including 
automated  hot  plates  and  various  types  of  infrared 
furnaces.  The  most  successful  results  to  date  have 
been  by  vapor  phase  reflow. 
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The  principle  of  vapor  phase  soldering  is  as 
follows:  A  fluorinert  material  is  heated  in  a  tank  to 
form  a  vapor  at  215°C;  coils  at  the  top  of  the  tank 
keep  the  vapors  from  escaping  by  reccmdensing  the 
vapor.  Parts  to  be  soldered  are  placed  on  a  tray  that  is 
controllable  both  in  q>eed  of  lowering  into  the  vapor 
and  dwell  time  in  the  vapor.  The  vapor  gives  up  its 
heat  of  vaporization  to  the  parts  by  condensing  on 
them;  soldering  is  rapid  and  is  controlled  to  a  maxi¬ 
mum  temperature  of  215°C.  Evaluation  of  this 
method  indicates  that  the  time/temperature  of  the 
metallization  process  is  minimum  and  the  best  con; 
trolled  of  all  soldering  systems  tested. 

Following  the  assembly  operation,  final  electrical 
testing  is  conducted.  Experience  at  RCA  indicates  a 
substantially  higher  yield  with  this  process  than  with 
complex  single-cavity  sealed  hybrids  (equivalent  to 
printed-circuit  board  assembly  of  tested  compo¬ 
nents).  In  case  of  chip  failure  after  assembly,  repair  is 
a  relatively  simple  matter  of  heating  and  removing  the 
chip  carrier  and  replacing  the  faulty  components. 
Since  repair  operations  do  not  require  unsealing  a 
large  hermetic  package  or  working  with  delicate  chips 
and  wire  bonds  under  a  microscope,  the  repair  can  be 
accomplished  by  trained  soldering  technicians. 

MATURITY  OF  CHIP  CARRIERS 

The  chip  carrier  has  provided  the  hi^-density 
packaging  engineer  with  an  alternative  to  sin|d«-cavity 
hybrids  that  in  many  cases  equals  the  conventional 
hybrid  packaging  density.  It  is  practical,  and  has  been 
proven  in  numerous  production  applications,  to 
mount  and  interconnect  chip  carriers  on  both  sides  of 
a  ceramic  substrate.  Although  the  chip  carrier 
occupies  about  twice  the  area  of  the  sin^e  cavity 
hybrid,  vrhen  both  sides  of  the  substrate  are  used  by 
chip  carriers,  the  packaging  density  is  equalled.  RCA 
has  been  using  this  capability  since  1976  when  the 
mirror  photograph  (Figurh  6)  was  taken  of  both  sides 
of  a  sample  hybrid.  Our  production  processes  have 
provided  excellent  yields  with  standard  techniques. 
The  method  offers  an  additional  advantage  of  more 
easily  tailoring  the  size  and  shape  of  the  ceramic  sub¬ 
strate  in  comparison  to  relatively  fixed  configurations 
of  sealed  hermetic  hybrid  packages.  This  advantage 
can  now  be  used  to  tailor  hybrid  size  and  shape  with 
chip  carriers  on  ceramic. 


They  are  produced  in  50  mil  centor-to-center  pan 
dimensions  and  40  mil  center-to-center  dimensions. 
The  JEDBC  family  designed  by  RCA  under  contract 
to  the  Air  Force  is  shown  in  Figure  7.  Note  that 
althou^  40  mil  carriers  are  preferred  for  minimum 
size,  either  40  or  60  mil  center-to-center  leads  can  be 
used  by  the  hybrid  manufacturer.  50  mil  center-to- 
center  chip  carriers  are  currently  preferred  for  appli¬ 
cations  for  printed  circuit  board  users  because  of  the 
match  to  dimensional  standards  of  25  mil  features 
and  100  mil  spaced  holes  for  printed  circuit  boards. 
Both  Hufdres  and  Texas  Instruments  have  been 
funded  by  DoD  for  50-mil  center-to-center  chip 
carriers  which  are  standardized  by  JEDEC. 


Figure  6.  Double  Sided  Hybrid 


APPUCATIONS 

Numerous  applications  have  been  developed  by 
RCA  vdiich  have  been  produced  for  both  commercial 
and  militarj)  environments.  Typical  examples  are 
shown  in  Figures  8  and  9.  Figure  8  shows  a  double 
sided  Navy  “Improved  Standard  Electric  Module” 
containing  70  chips.  Thirty-two  IK  memory  chips 
jHuf  four  control  chips  are  mounted  and  inter¬ 
connected  by  multilayer  thick-film  ceramic  substrates 
on  each  side  of  the  module.  Figure  9  shows  a  hi^- 
speed  16-bit  microprocessor  packaged  in  the  new 
'JEDEC  standardized  84-lead  chip  carriers  with  high 
speed  multiplier  chips  in  64-lead  chip  carriers.  Chips 
in  low-power  high-speed  CMOS/SOS,  and  module 
packaging  are  provided  by  RCA. 


In  addition  to  the  wide  variety  of  chip  carriers 
available  commercially,  an  active  standardization  pro¬ 
gram  has  been  undertaken  with  help  from  DoD:  the 
result  is  two  families  of  chip  carriers  suitable  for  auto¬ 
mated  production  tooling  have  been  approved  by  the 
Joint  Electron  Device  Engmeering  Council  (JEDEC). 


F^re  10  shows  an  assembly  of  components  in 
chip  carriers,  attached  and  interconnected  by  multi¬ 
layer  thick  film  on  ceramic,  and  then  interconnected 
together  by  a  four-layer  PCB.  Note  that  one  unit  is 
attached  directly  to  the  PCB  by  the  same  stress 
relieving  pins  used  to  attach  the  ceramic  substrates. 


Figure  7. 


40  Mil  Center-to-Center  Family  of  Chip  Carriers  Proposed  for  JEDEC  Standardization 


Multiplier 


Figure  10.  Printed  Circuit  Board  with  Groups  of  Leadless 
Ceramic  Chip  Carriers  on  Ceramic  Sub-boards 
and  Leaded  Chip  Carrier  on  PCB 

on  multilayer  ceramic  substrates  to  provide 
packing  densities  comparable  to  conventional 
“chip  and  wire”  hybrids. 

4.  HCC  maturity  has  been  “accelerated”  by  DoD 
investment  and  industry  activity  to  standardize 
via  JEDEC  JC-11.3.1  chip  carrier  outlines, 
footprints,  and  mechanical  features  for 
orientation  and  alignment,  and  burn-in  and 
test  sockets. 


CONCLUSIONS 

1.  LSI  and  VLSI  technology  with  many  I/Os 
requires  a  new  packaging  technology  so  that 
the  inherent  advantages  of  increased  per¬ 
formance  and  reduced  size  at  the  chip  level  can 
be  realized  at  the  package  and  module  levels. 

2.  Increased  complexity  of  LSI  and  VLSI 
requires  complete  testing  of  all  I/Os  and  at 
temperature  limits  for  high  reliability 
applications. 

3.  HCCs  improve  performance  by  reducing  lead 
capacitance  and  conductance,  reduce  size  to 
one  fifth,  and  provide  a  mechanism  for  test 
and  handling.  In  fact,  it  is  possible  with  HCCs 


5.  Techniques  have  been  established  and  proven 
for  reliable  assembly  and  interconnections  on 
ceramic  substrates  and,  with  stress  relief,  on 
organic  printed  circuit  boards. 

RTCOMMENDATIONS 

1.  Reduced-size  chip  carriers  are  indicated  for 
future  use  of  48  I/O  and  larger  HCCs. 

2.  Automated  module  assembly  techniques  and 
tooling  are  required  to  reduce  production 
module  costs. 

3.  Commercial  availability  of  components  in  chip 
carriers  is  required  to  increase  production  and 
reduce  cost. 
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VALIDATION  AND  VERIFICATION  OF  LSI 
TECHNOLOGIES  AND  DESIGN  TOOLS 

C.  J.  VanLeeuwen,  Manager,  Reliability 
Harris  Semiconductor,  Products  Division 


Introduction 

At  Harris  Semiconductor  reliability  is 
considered  a  performance  parameter, 
encompassing  both  device  and  process 
technology,  whose  time  constant  is  long 
term.  To  accomplish  this,  reliability 
representation  during  design  and  layout 
reviews  of  any  new  product  type  of  new 
technology  is  critical  and,  therefore, 
mandatory.  It  is  during  early  phases 
of  new  circuit  development  when  employ¬ 
ing  a  new  technology,  that  reliability 
based  layout  ground  rules  are  invoked 
to  ensure  that  reliability  is  designed 
and  built  in  rather  than  only  verified 
after  the  product  technology  phases  are 
finalized. 

To  facilitate  timely  release  of  new 
LSI  devices,  deploying  a  brand  new  pro¬ 
cess  technology,  specially  designed 
test  vehicles  are  first  constructed  to 
characterize  pertinent  process  para¬ 
meters.  Going  one  step  further  in  the 
prove-in  phase  of  a  new  technology,  for 
example  a  LSI  memory  circuit,  computer 
controlled  design  ground  rules  are 
implemented  to  accommodate  computer 
aided  generation  of  all  mask  sets  using 
a  bit-map  topological  checking  system. 

Reliability  Prove-in 

Central  to  successful  reliability  prove- 
in  of  our  new  product  lines  are  the 
following  steps  (see'Figure  1). 

Step  1.  Early  mandatory  reliability 
interfacing  during  the  design 
and  layout  reviews  of  future 
circuits  with  reliability 
ground  rule  compliance. 

Step  2.  Reliability  representation  and 
consultation  during  concept 
and  deployment  phases  of  new 
process  technologies. 

Step  3.  Reliability  representation  with 
qualification  requirements  as  a 


prerequisite  to  transfer  new 
product  into  volume  production. 

Step  4.  Final  reliability  verification 
of  the  first  run  of  a  newly 
•  transferred  product  type  from 

volume  production.  Subsequent 
reliability  qualification  will 
result  in  release  to  ship. 

The  above  steps  are  all  equally  critical 
in  ensuring  that  reliability  is  built  in 
and,  at  the  same  time,  to  safeguard 
against  costly  redesign  delays. 

Before  advancing  a  new  technology,  our 
need  for  specially  designed  test 
vehicles  is  reviewed.  If  found  neces¬ 
sary  for  reliability  characterization 
of  the  new  process,  test  vehicles  are 
deployed  first  before  product  relia¬ 
bility  evaluation  commences.  (See 
Figure  2. ) 

In  so  doing,  the  inherent  reliability 
of  the  new  process  technology  is 
determined  first  to  avoid  potential 
problems  in  later  stages. 

Follow-on  In-depth  New 
Technology/Product  Characterization 

Following  familiarization  of  circuit 
design,  chip  layout  topography,  and 
product  functional  operation  (deploying 
a  new  technology),  samples  from  the 
first  development  run  are  received  for 
reliability  evaluation.  This  evaluation 
encompasses  the  following  rigorous  tests 
and  checks:' 

1.  Stressing  devices  to  destruction  by 
exceeding  maximum  allowable  voltage 
and  temperature  to  determine  failure 
modes/mechanisms  with  respect  to 
degree  of  existing  safety  margins 
between  applied  normal  operating 
stress  conditions  and  absolute 
maximum  ratings.  This  test  provides 
insight  into  the  inherent  weakness 
of  the  design  and  coupled  new 
technology. 

2.  Determine  sensitivity  to  ESD  which 
is  especially  critical  to  MOS  and 
small  geometry  bipolar  components. 
Sensitivity  to  ESD  is  not  considered 
a  technology  and/or  design  weakness 
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as  precautionary  handling  measures 
are  available  {see  references  1,  2, 
3).  Also,  design  techniques  can  be 
used  to  minimize  ESD  effects. 

3.  Ensure  at  the  wafer  level,  prior  to 
the  application  of  passivation,  that 
coverage  of  metallization  over 
(steep)  oxide  steps  is  complete  with 
no  "rat-holing"  problems. 

4.  Perform  life  testing  for  1000  hours 
minimum,  dynamic  and  static,  with 
intermediate  variables  data  read¬ 
outs  at  168  and  500  hours  at  nominal 
VCC  (Vdd)  conditions  at  1250C 
ambient  with  all  failures  being 
analyzed.  Acceptance  criteria  at 
this  state  of  the  evaluation  must 
always  be  clearly  defined.  To  pro¬ 
ject  high  temperature  related 
failure  rates  to  end-use  operating 
conditions,  derating  by  means  of 
the  Arrhenius  reaction  rate  is 
accomplished.  This  is  predicated 
on  process  dependent  activation 
energy  values  (such  as  0.3  to  0.6 
eV  for  CMOS/Bipolar  —  corrosion, 
aluminum  electromigration,  contamin¬ 
ation,  etc.;  1.00  to  1.15  eV  for 
Bipolar  —  surface  instability; 

1.12  to  1.35  eV  for  CMOS  —  surface 
instability).  The  latter  are  either 
obtained  from  the  literature  or 
emperically  determined. 

The  additional  tests  below  are  more 

directly  product  related: 

1.  Verification  of  residual  moisture 
content  internal  to  the  package  by 
means  of  the  dew  point  technique 
using  the  Harris  HIO-55001-6  mois¬ 
ture  sensor  chip  which  is  periodi¬ 
cally  calibrated  against  a  reputable 
mass  spectrograph! c  analysis. 

2.  Thermal  impedance  evaluation  (0j-a 
and  9j-c). 

3.  Internal  visual  scrutiny  for  possible 
anomalies  pre  and  post  stressing 
which  have  not  progressed  far  enough 
to  cause  malfunction. 

4.  For  epoxy  sealed  products  key  tests 
are: 

•  168-hour  autoclave  (with  no  bias). 


a  1000-hour  850C/85%  RH  with'  bias, 
a  Temp-cycle, 
a  Thermal  shock. 

a  Continuously  monitored  temp-cycle 
to  check  for  intermittency. 

5.  As  a  stand-alone  procedure,  all 

vendor  package  types  and  piece  parts 
are  qualified  in  accordance  with 
rigorous  MIL-STD-883B  procedures  to 
ensure  that  thermal  and  mechanical 
environmental  conditions  are  met 
initially  and  continue  during  on¬ 
going  monitoring  programs.  This  also 
includes  internal  particle  detection 
evaluation. 

Examples  of  Process  Technology 
Evaluation 

We  have  learned  that  when  choosing  a 
technology,  you  have  to  design  a  cir¬ 
cuit  around  it  to  avoid  raising  a 
conflicting  capability  later  while  try¬ 
ing  to  match  the  technology  with  certain 
circuit  performance  parameters.  This 
has  resulted  in  the  concept  of  evaluat¬ 
ing  process  test  vehicles. 

Specially  designed  test  vehicles  and/or 
expander  dice  were  used  to  reliability 
qualify  the  following  new  process  tech¬ 
nologies  prior  to  circuit  (device) 
deployment. 

1.  Nichrome  Fuse  Technology  for 
Bipolar  PROMs 

When  the  first  (bipolar)  PROM  in  the 
industry  was  announced  by  our  com¬ 
pany,  the  0512-HPR0M  was  introduced 
deploying  nichrome  fusible  links  to 
program'  the  bit  content  contained 
in  the  memory  matrix.  Prior  to  the 
release  of  the  first  PROM,  a  relia¬ 
bility  study  was  performed  to 
determine  the  optimum  fusing  condi¬ 
tions;  i.e.,  progranmabi lity,  pulse 
width,  fuse  geometry  with  minimum 
applied  voltage,  etc.,  as  a  require¬ 
ment  to  ensure  that  programmed  fuses 
remain  open  (and  conversely, 
unprogratimed  fuses  always  remain 
intact)  during  end-use  operation  of 
the  PROM.  This  study  enabled  us  to 


determine  the  critical  circuit  para¬ 
meters  afforded  in  the  PROM  to 
accommodate  the  ni chrome  fuse  as  a 
viable  and  reliable  technology. 
(Reference  4.) 

Elaborating  on  this,  the  key  to 
successful  reliability  prove-in  of 
the  ni chrome  fusible  link  was  the 
ultimate  understanding  about  the 
fusing  mechanism.  After  having 
ruled  out  the  first  three  (A,  B,  C) 
possibilities  below,  we  found  that_ 
the  fusing  force  obeys  the  field 
enhanced  ionic  mass  transport 
mobi 1 i ty  model  (D) . 

A.  Electromigration;  Mass  flux 
occurs  under  the  influence  of 
high  current  flow  because 
electron  collisions  with  atoms 
of  the  conducting  medium  pro¬ 
vide  a  net  motion  vector  in  the 
direction  of  electron  flow. 

B.  Thermal  gradient;  In  the  pre¬ 
sence  of  a  thermal  differential, 
material  will  diffuse  from  the 
high  temperature  to  the  cold 
temperature  region. 

C.  Concentration  gradient;  In  an 
imbalanced  distribution  of 
concentration  to  lower  concen¬ 
tration. 

0.  Field  enhanced  ionic  mobility 
(Reference  5);  Molten  metals 
will  ionize,  lose  electrons  and 
become  cations.  In  the  pre¬ 
sence  of  an  electric  field, 
they  will  be  driven  towards  the 
cathode. 

A  computer  tnermal  analysis  program 
called  "THEROS"  was  used  to  calcu¬ 
late  the  dynamic  temperature  effects 
in  a  PROM-fuse  structure  as  a 
function  of  applied  power  density. 
The  expression  "power  density"  is 
defined  as  the  amount  of  power  that 
is  dissipated  in  the  fuse  neck 
region  divided  by  its  cross- 
sectional  area.  The  concept  of 
defining  power  density  as  power  per 
unit  surface  area  is  applicable  to 
thin  film  heat  flow  problems  where 
the  heat  is  dissipated  through  a 
surface.  (The  concept  is  analogous 


to  defining  current  density  as  cur¬ 
rent  per  cross-sectional  area.) 

With  reference  to  Figure  3,  a  plot 
of  computer  results  shows  the  tem¬ 
perature  in  the  center,  narrow  neck 
region,  can  easily  reach  the  melt 
temperature  of  ni chrome  within 
microseconds  for  power  densities 
>2.5  watts/mil2. 

Figure  4  is  a  plot  of  the  intercept 
of  the  time  to  reach  the  melt  tem¬ 
perature  (1450OC)  versus  the  power 
density.  This  theoretical  predic¬ 
tion  of  the  power  density  versus 
time  to  reach  the  melt  temperatures 
compares  well  with  experimental 
data  on  time  to  fuse.  The  data  in 
Figure  4  was  taken  from  the  test 
vehicle  fuses  (processed  identi¬ 
cally  to  circuit  fuses,  but  free  of 
interfacing  circuitry).  This  allow¬ 
ed  precise  characterization  of  fuse- 
pulse  interactions.  The  data 
matches  for  long  fusing  time  but 
deviates  for  short  fusing  times. 

This  difference  can  be  accounted 
for  by  considering  the  definition 
of  "time-to-fuse."  The  experimental 
data  points  represent  total  time  to 
fuse  which  includes  rise  time  of 
the  programming  pulse,  time  for  the 
fuse  to  heat  to  sufficient  tempera¬ 
ture,  and  time  of  the  actual  fusing 
event.  For  example.  Figure  5  shows 
a  typical  current  trace  for  a  fuse 
programmed  under  constant  voltage 
conditions.  The  trace  shows  a 
fixed  rise  time,  tr  (about  100 
nanoseconds  for  this  data),-  a 
response  time,  tm,  for  the  ni chrome 
to  reach  the  melt  temperature,  and 
a  time  for  the  fuse  neck  to  enter 
the  melt  phase  and  program,  tf. 
Plotting  the  time  defined  as  tm 
shows  excellent  correlation  with 
the  theoretical  prediction  of  the 
time  to  reach  melt  temperature. 

The  difference  between  the  theoret¬ 
ical  prediction  to  reach  melt  and 
the  actual  time  to  fuse  agrees  with 
the  measured  values  of  tr  +  tf. 
Figure  4,  therefore,  shows  that 
fusing  follows  a  heat  flow  depend¬ 
ence  that  requires  the  ni chrome  to 
achieve  melt.  Proper  PROM  design 
necessitates  taking  into  account 
thermal  factors  that  affect  the  heat 
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flow  conditions  in  the  neighborhood 
of  the  fuse.  Concentrating  power  by 
optimum  fuse  geometry  and  ensuring 
sufficient  power  to  the  fuse  will 
achieve  fast,  uniform  programming. 

We  found  that  the  power  density 
versus  time  to  program  curve 
(Figure  4)  is  described  well  by 
the  heat  flow  model  and  implies  a 
single  mechanism,  melting,  for 
fusing  both  fas't  and  slow  program¬ 
ming.  Further,  high  power  program¬ 
ming  (fast  blow)  approaches 
adiabatic  heating  conditions  and 
therefore  gives  a  large  melted 
region  and  a  wide  gap;  restricted 
power  programming  (slow  blow)  on 
the  other  hand  allows  much  of  the 
heat  to  diffuse  away  taking  longer 
for  the  fuse  to  reach  melt. 


We  also  learned,  by  grossly  violat¬ 
ing  recommended  programming  proce¬ 
dures  for  fuses,  that  it  is  possible 
to  create  a  marginal  fuse  gap  that 
may  be  subject  to  reverting  state 
("grow-back").  This  anomaly  was 
induced  in  a  test  vehicle  fuse  by 
restricting  the  power  input  to  a 
value  on  the  t->oo  asymptote  ('^1.5 
watts/mi l2)  of  the  power  density 
versus  time  to  fuse  curve  (see 
Figure  4).  Under  these  conditions, 
a  fuse  was  induced  to  program, 
become  electrically  discontinuous, 
after  5  minutes  of  sustained  power. 
This  effect,  programming  under 
anomalously  reduced  power,  was  not 
found  to  be  reproducible.  Many 
fuses  at  this  power  density  would 
not  program  after  days.  During  this 
study  we  also  found  that  the  fusing 
phenomenon  based  on  SEM  results  have 
been  erroneous  or  misleading  because 
what  was  seen  was  an  artifact  of 
sample  preparation.  The  technique 
of  using  transmission  electron 
microscopy  (TEM)  to  examine  a  pro¬ 
grammed  fuse  gap  (see  Figure  6)  is 
the  only  technique  which  mutually 
satisfies  the  requirements  of  suf¬ 
ficient  resolution  to  analyze  the 
gap  and  not  destroy  in  sample  pre¬ 
paration  the  original  structure  to 
be  analyzed.  Also,  we  found  that 
in  depassivating  devices,  fuse  gaps 
are  destroyed. 


A  deliberately  improperly  programmed 
fuse  was  subsequently  subjected  to  a 
slowly  applied  DC  voltage  ramp  under 
current  limited  conditions  (lOM-ohm 
resistor  in  series).  At  12  volts  the 
fuse  resistance  dropped  tO'^5,000 
ohms.  The  TEM  photograph  of  this 
fuse  is  shown  in  Figure  7.  It  is 
obvious  from  this  photograph  that  the 
reduced  power  condition  has  resulted 
in  a  fuse  that  has  marginally  pro¬ 
grammed.  That  is,  the  gap  created 
after  programming  is  very  narrow 
(approximately  a  few  hundred  ang¬ 
stroms)  and  subject  to  a  voltage 
breakdown  effect. 

Fuses  programmed  per  the  recommended 
power  levels  will  program  rapidly 
with  a  wide  gap  in  agreement  with 
the  mass  transport  fusing  model. 

These  fuses  can  be  subjected  to  more 
than  100  volts  and  will  undergo  no 
change  in  electrical  or  physical 
condi tion. 

Additionally,  as  in  Figure  7,  if  a 
restricted  amount  of  power  is  applied 
to  a  fuse,  it  is  possible  to  create 
a  very  narrow  gap.  Under  the  pre¬ 
sence  of  high  voltage  and  extreme 
current  limiting,  it  is  then  possible 
to  force  a  voltage  breakdown  across 
the  gap.  It  is  postulated  that  this 
voltage  discharge  results  in  the 
establishment  of  a  low  conductivity 
relink  at  one  or  a  few  points  of 
closest  approach  in  the  marginally 
blown  gap.  This  specific  structure 
could  not  be  confirmed  with  the  TEM 
study  because  even  the  TEM  did  not 
have  the  resolution  to  examine 
microstructures  at  <300  angstroms. 

• 

2.  On-board  MOS  Capacitors  for  Linear 
Devices 

MOS  capacitors  were  reliability 
qualified  as  stand-alone  components 
with  respect  to  time- temperature 
applied  voltage  exposure  and  maximum 
voltage  surge  to  check  susceptibility. 
Final  reliability  prove-in,  which 
also  allowed  us  to  tighten  critical 
process  controls  (e.g.,  dielectric 
oxide  integrity),  led  to  the  success¬ 
ful  deployment  of  the  MOS  capacitor 
in  linear  operational  amplifiers  such 
as  the  HA-2600,  2620,  etc. 


3. 


I^ysilicon  Fuse  Technology  for  CMOS 
PROMs 


When  the  polysilicon  fuse  technology 
emerged,  as  another  viable  program¬ 
mable  fuse  technology,  several  years 
after  the  ni chrome  fusible  link  PROM 
made  its  successful  debut,  aspects 
similar  to  the  nichrome  fuse  techno¬ 
logy  were  visioned.  Considerable 
advantage  was  taken  of  the  compati¬ 
bility  of  our  company's  SAJI  CMOS 
RAM/ROM  process.  SEM  examinations 
were  also  made  to  correlate  with 
the  electrical  results  which  clearly 
showed  that  programmed  and  unpro¬ 
grammed  fuses  are  rel labile  with  no 
latent  problems.  lightning  of 
critical  process  controls  was  found 
necessary  which  led  to  improved 
programmabi 1 i ty  yields.  It  should 
be  mentioned  that  the  test  fuses 
of  each  CMOS  PROM  are  100'  pro¬ 
grammed  at  final  test  (as  is  the 
case  in  nichrome  fuse  bipolar 
PROMs).  This  also  includes  100', « 
testing  the  corresponding  test  rows 
and  columns.  This  provides  the 
ultimate  assurance  that  the  user 
does  not  have  to  fear  marginal  pro¬ 
gramming  due  to  residual  undetect¬ 
able  process  flaws. 


Via  Contact  Technology  to  Link 
Multi-level  Metallization  Systems 


With  the  advent  of  reduced  die  size, 
complex  1C  chips  had  to  resort  to 
the  use  of  a  multi-level  metalliza¬ 
tion  process  requiring  a  reliable 
level-to-level  interconnect  system. 
To  understand  this  critical  connec¬ 
tion,  special  via  test  vehicles 
were  designed  and  produced  to  under¬ 
go  subsequent  reliability  qualifi¬ 
cation.  (Reference  6) 


Pertinent  results  of  the  reliability 
prove-in  of  the  via  contact  techno¬ 
logy  were  as  follows.  Two  specially 
designed  two-level  aluminum  metal¬ 
lization  test  structures  were 
deployed.  Each  structure  contained 
fifty  via  contacts,  one  being  the 
0.25-mil  via,  and  the  other  the 
0.50-mil  via.  Both  were  incorpor¬ 
ated  on  a  single  silicon  chip. 

Figure  8  represents  a  close-up  view 
of  a  typical  level-to-level  via 


contact  structure  showing  the  first 
and  second  level  metal  being  inter¬ 
connected  by  a  via  contact  in  an 
interwoven  fashion,  thus  forming  a 
complete  chain  of  fifty  via  contacts 
connected  electrically  in  series. 
Figure  9  is  a  cross-sectional  dia¬ 
gram  of  the  passivated  two-level 
metal  test  structure  showing  the  via 
contacts.  (The  unpassivated  via 
contact  vehicle  has  no  protective 
Si02  over  the  second  level  aluminum 
metallization.)  The  aluminum  thin 
film  cross-sectional  area  measures 
typically  4.25  x  lO'^cni^.  For 
testing,  the  chips  containing  the 
two  test  structures,  were  mounted, 
wire  bonded  and  sealed  in  a  4-lead 
TO-5  package. 

In  executing  the'  applied  stress 
evaluation  program  we  learned  that 
the  0.50-mil  via  contacts  were 
better  by  1  x  105A/cm2  in  stress 
attainability.  Interesting  SEM 
observations  were  made  during  a 
step-stress  test  as  can  be  noted  in 
Figure  10. 


Stable  current  stressing  was  provid¬ 
ed  by  'means  of  a  constant  current 
source  with  consistent  polarity 
during  all  tests.  Figure  11  shows 
the  step-stress  results.  Failure 
incidence  is  defined  as  electrical 
discontinuity  anywhere  along  the 
chain  of  the  fifty  via  contacts  per 
test  vehicle.  As  can  be  noted,  the 
503  failure  point  of  passivated  via 
contacts  (0.25-mil  and  0.50-mil  vias 
combined)  occurred  at  a  current 
density  of  about  12  x  105A/cm2  as 
opposed  to  about  9  x  105A/cm2  for 
via  contacts  with  unpassivated  second 
level  metal.  This  difference  is 
indicative  of  improved  reliability 
performance  of  passivated  via  con¬ 


tacts.  In  addition,  the  0.50-mil 
via  contacts  were  better  by  1  x  10^ 
A/cm2  in  stress  attainability  at  the 
50'„  failure  point  than  the  0.25-mil 


via.  Close  examination  of  the 


failures  by  means  of  the  SEM  showed 
evidence  of  electromigration  with 
formation  of  hillocks  and  voids  in 


opposing  via  contact  pairs  as  shown 
in  Figure  10.  This  failure  phenome¬ 
non  was  consistent  during  both  step- 
stress  and  constant  accelerated 
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stress  testing.  No  other  failure 
modes  occurred. 

To  predict  useful  life  of  the  0.25- 
mil  via  contact  two-level  metal 
system,  three  randomly  selected 
samples  of  ten  vehicles  were 
stressed  at  different  accelerated 
current  densities:  7.05  x  lO^A/cm^, 
6.47  X  105A/cm2,  and  4.47  x  lO^A/ 
cm2.  In  each  case,  the  ambient 
temperature  was  maintained  at  +1250C. 
The  resulting  plot  in  Figure  12 
shows  the  projection  to  in-use  con¬ 
ditions  of  the  50/  failure  data  as 
generated  by  the  three  sample 
failure  distributions.  A  log-normal 
failure  behavior  was  noted  for  each 
of  these  distributions.  The  common 
failure  mode  was  electromigration 
showing  alternate  void  and  hillock 
formation  in  the  via  contacts  result¬ 
ing  in  electrical  discontinuity.  At 
a  typical  in-use  current  density 
condition  of  1  x  10^A/cm2  for  M/LSI 
circuits,  employing  two- level 
metallization  via  contacts,  the  plot 
shows  an  estimated  mean  time  to 
failure  in  the  order  of  7,000  hours. 
This  was  unsatisfactory  and  0.25- 
mil  aluminum  via  contacts  were 
therefore  not  considered  for  circuit 
ap()l  i  cation.  Similar  data  for  the 
0.5()-mil  via  contact  vehicle 
resulted  in  85,000  hours  of  extra¬ 
polated  mean  life. 

fSD,  LOS  and  Latch-u])  Sensitivity 

Although  LSD  (Llectrostatic  dis- 
char(je)  and  over’voltage  conditions 
are  not  considered  inherent  device 
reliability  prolilems,  they  play  an 
important  part  in  choosing  a  new 
Lectinology  as  it  applies  to  new 
circuit  development.  For  example, 

1  SU  and  latch-up  sensitivity  are 
not  specified  as  guaranteed  para¬ 
meters,  yet  they  can  make  the 
difference  in  customer  acceptance. 

The  particular  characteristics  of 
immunity  to  LSD  and  latch-up  must 
be  d('sigtuHl  in.  This  reciuires 
('valuation  of  test  vehicles 
retlectiiKi  the?  ground  rules  which 
would  ('nhanc('  the  above  immunity, 
in  !ie((  of  usini)  the  end-product  as 
ail  evaluation  vi'hicle.  Here 
('‘.pe(_  i a  1  I y  exist,  the  danger  of  over¬ 


looking  the  growing  tendency  to 
sacrifice  reliability  safety  margins 
for  topographic  densi f ication  by 
lessening  spacings  and  routings  for 
electrical  charge  to  achieve  faster 
(AC)  performance,  thus  compelling 
to  push  the  limits  of  a  technology. 

6.  Alpha  Particle  Sensi ti vi ty 

There  exists  a  general  awareness 
about  alpha  particle  sensitivity  in 
that  it  can  affect  memory  bit 
integrity  among  MOS  devices.  An 
extensive  internal  study  and  an 
industry-wide  fact  finding  litera¬ 
ture  search  was  made  from  which 
conclusions  were  drawn  that  CMOS  is 
the  least  sensitive  to  the  problem. 

7.  Yx  Expander  [^ij;_e 

Relevant  to  Vj  evaluation,  a  new 
technology  expander  die  allows 
monitoring  permissible  drift  under 
applied  stress  conditions.  To  meet 
high  temperature  stability  depends 
on  how  well  we  can  control  a  process 
with  respect  to  locking  out  conta¬ 
mination  with  no  loss  in  yields. 

As  an  integral  part  of  reliability  qual¬ 
ifying  new  process  technologies  and  new 
complex  ICs  utilizing  these  technologies, 
extensive  use  is  made  of  the  bit-map 
checking  system  at  Harris. 

The  bit-map  topological  checking  system 
for  processing  IC  mask  sets  covers  all 
geometric  topographic  circuit  layout 
aspects  (polygons)  as  it  relates  to  the 
diffusions,  multilevel  metallization 
(polysilicon  and  aluminum),  passivation, 
etc.  This  applies  to  any  IC  technology 
of  varied  complexity.  This  system  can 
check  component  densification  and  in  so 
doing  obey  earlier  programmed  reliability 
oriented  design  ground  rules.  Any  ground 
rule  violation  is  readily  detectable. 

Figure  13  illustrates  our  typical  basic 
bit-map  setup  in  functional  block  diagram 
format. 

Typical  polygon  operators  are: 

•  contains 


•  contained 


touch 


•  inside  touch 

•  outside  touch 

•  overlap 

•  coincide 

•  disjoint 

•  fill 

All  these  allow  control  of  geometric 
process/circuit  topologic  coordination. 


What  Has  Been  Learned 

The  foregoing  approach  to  the  relia¬ 
bility  qualification  of  new  technologies 
has  provided  us  with  experience  from 
which  the  following  key  recommendations 
can  be  made: 

1.  Utilize  test  vehicles  to  qualify 
new  process  technology, 

2.  Implement  reliability  based  ground 
rules  from  design  on  out  to  ensure 
reliability  is  built  in. 

3.  Determine  how  a  LSI  circuit  fails 
via  overstressing  in  relation  to 
possible  process  technology  weak¬ 
nesses, 

4.  Perform  defect  analysis  on  fallout 
from  wafer  probe  and  final  test  to 
enhance  yields  by  eliminating 
defect  incidence. 

The  above  approach  hat  been  adopted  in 
our  reliability  procedures  as  a  routine 
matter  and  by  proving  in,  for  example, 
new  technologies  as  referred  to  earlier, 
the  following  conclusions  are  drawn: 


1.  Stressing  product  to  destruction 
has  always  allowed  us  insight  into 
basic  strength  and  weaknesses  of  a 
technology. 

2.  Defect  analysis  for  yield  enhance¬ 
ment  purposes  has  shown  us  where  to 
tighten  the  controls  in  the  process 
flow  as  well  as  what  latent  problems 
to  expect  later  in  the  field  if  we 
would  not  make  corrections. 

3.  Implementation  of  reliability  ground 
rules  has  always  paid  off  dividends 
in  that  costly  redesigns  were  avoided 
with  reliability  pay- back. 

What's  Ahead 

What's  ahead  is  actually  already  here 
when  the  advanced  failure  analysis  section 
of  the  reliability  department  at  our  com¬ 
pany  performs  routinely  in-depth  defect 
diagnosis,  with  the  aid  of  sophisticated 
analytical  apparatus  such  as  SEM,  SAM 
and  SIMS,  electron  microprobe,  etc.,  on 
wafer  probe  and  final  test  IC  fallout. 

This  is  in  support  of  yield  enhancement 
for  the  various  product  engineering 
groups.  What  this  also  provides  is 
important  feedback  to  the  new  technology 
groups.  This  is  in  terms  of  intolerable 
material  defect  levels,  possible  process 
induced  contamination  and  flaws  with 
emphasis  on  the  associated  latent  char¬ 
acter  as  it  can  adversely  affect  future 
technologies.  It  will  also  lead  to  the 
further  refinement  of  the  analytical 
tools  and  methods  to  rapidly  perform 
defect  diagnosis  on  state-of-the-art 
technologies  and  their  highly  complex 
large  scale  IC  derivatives.  At  Harris 
we  intend  to  continue  to  face  that 
challenge  of  coupling  sophisticated 
evaluation  procedures  with  diagnostic 
means  and  computer-aided  methods  to 
cover  ICs  of  ever  increasing  density  and 
complexity  employing  advanced  high 
technologies. 
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GALILEO  ORB I TER  COMMAND 
AND  DATA  SUBSYSTEM 

Wayne  H.  Kohl 
Jet  Propulsion  Laboratory 
Pasadena,  California  91103 

General 

The  1984  launch  Galileo  mission  consists 
of  two  separate  shuttle  launches,  the 
probe  with  carrier  and  the  orblter.  Tha 
gallleo  orblter  is  to  some  extent  an  evo¬ 
lution  of  previous  Jet  Propulsion  Labora¬ 
tory  spacecraft  except  for  marked  .differ¬ 
ences  such  as  three  axis  stabilization 
versus  a  dual  spinner.  Another  difference 
is  the  Command  and  Data  Subsystem  (CDS). 
This  subsystem  evolved  from  the  combina¬ 
tion  of  two  special  purpose  central  com¬ 
puters,  the  Computer  Command  Subsystem  and 
the  Flight  Data  Subsystem.  Its  tasks  in¬ 
clude  decoding  uplink  command  messages, 
storage  and  execution  of  timed  commands, 
collection  and  formatting  of  dovmllnk 
telemetry,  and  the  detection  and  correct- 
tion  of  spacecraft  errors. 

In  order  to  obtain  a  better  understanding 
of  the  CDS  design  one  must  understand  its 
role  on  the  spacecraft.  Figure  1  is  a  CDS 
view  of  the  spacecraft  and  how  the  other 
subsystems  and  systems  Interact  with  it. 

As  can  be  seen  in  the  figure,  the  uplinked 
data  passes  through  the  S/X  Band  Antenna 
Subsystem,  Radio  Frequency  Subsystem,  and 
Modulation  Demodulation  Subsystem,  arriv¬ 
ing  at  the  CDS  as  an  NRZ  binary  data 
stream.  It  is  the  CDS  task  to  synchronize 
to  the  data,  extract  the  command  informa¬ 
tion,  and  initiate  the  decoded  response. 
That  response  can  consist  of  passing  a  sub¬ 
set  of  the  data  to  another  subsystem  as  a 
command,  executing  the  command  if  it  is  for 
the  CDS,  or  storing  the  data  for  future 
evaluation  if  it  is  a  stored  sequence. 
Stored  sequences  take  the  form  of  time- 
event  tables  which  are  stored  temporarily 
in  the  CDS  until  a  specified  future  time 
is  realized.  The  events  are  then  inter¬ 
preted  and  can  range  from  simple  commands 
to  the  CDS  or  other  subsystems,  to  more 
complex  sequence  expansions  which  consist 
of  parameterized  routine  calls.  Those  rou¬ 
tines  can  cause  commands  to  be  issued  or 
initiate  other  stored  sequences. 

Besides  serving  as  the  central  command  dis¬ 
tributor,  the  CDS  is  also  responsible  for 


data  collection,  evaluation,  formatting, 
and  routing.  The  collected  data  consists 
of  subsystem  and  system  status  informa¬ 
tion,  science  observation  data,  support 
engineering  data,  and  memory  readout  data. 
Most  of  this  data  is  collected,  formatted, 
and  routed  to  the  telemetry  output  data 
ports  to  be  recorded  or  downlinked  from 
the  spacecraft.  Some  of  this  data,  how¬ 
ever  is  examined  by  the  CDS  in  order  to 
evaluate  the  spacecraft  performance  and 
state.  Improper  performance  or  erroneous 
states  are  generally  handled  by  initiat¬ 
ing  correction  responses  which  consist  of 
previously  stored  time-event  tables. 

Besides  the  command  and  data  handling 
functions,  the  CDS  serves  as  the  central 
spacecraft  timing  source  and  clock.  The 
clock  information  is  distributed  to  those 
subsystems  needing  it  via  the  command  and 
data  bus.  This  bus  represents  the  cen¬ 
tral  communication  path  between  the  dis¬ 
tributed  processors  of  the  various  sub¬ 
systems  and  it  is  the  CDS  responsibility 
to  maintain  the  agreed-upon  bus  protocol. 
It  is  also  the  CDS  responsibility  to  pro¬ 
vide  a  block  redundant  design  for  long 
life  reliability  and  the  ability  to 
actively  use  that  redundancy  to  provide 
single  fault  tolerance  during  critical 
mission  phases. 

Hardware  Description 

The  architecture  of  the  CDS  is  based  on  a 
distributed  approach  developed  at  the  Jet 
Propulsion  Laboratory  under  NASA  Contract 
NAS  7-100.^  This  architecture  was  selec¬ 
ted  to  take  advantage  of  the  new  micro¬ 
processor  technology  and  to  also  serve  as 
the  core  of  the  anticipated  distributed 
microprocessor  driven  spacecraft.  The 
design  selected  was  a  distributed  micro¬ 
processor  system  interconnected  by  a  high 
speed  data  bus  which  is  extended  to 
interface  with  the  other  microprocessor 


The  Unified  Data  System:  A  distributed 
Processing  Network  for  Control  and  Data 
Handling  on  a  spacecraft.  Proc.  IEEE 
National  Aerospace  and  Electronics  Con¬ 
ference,  NAECON,  Dayton,  Ohio,  May  1976 
(Rennels,  D.  A.,  Riis  Vestergaard,  B. , 
and  Tyree,  V.  C.) 

2 

RCA  1802  Microprocessor:  User  Manual 
for  the  CDP1802  COSMAC  Microprocessor 
(MPM-201B) 
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driven  subsystems.  The  extension  of  this 
bus  provides  the  primary  communications 
path  between  the  CDS  and  the  other  sub¬ 
systems/instruments.  As  the  Orbiter  con¬ 
sists  of  a  spun  and  despun  section,  this 
extension  must  pass  through  the  spin  bear¬ 
ing  assembly  to  link  the  spacecraft  sec¬ 
tions  together. 


which  can  be  assigned  to  any  bus  by 
the  critical  controller  and  extensive 
engineering  input/output  (I/O).  That 
I/O  Includes  digital  command  buffers, 
discrete  command  buffers,  analog  (in¬ 
cludes  temperature)  Inputs,  bi-level 
Inputs,  serial  digital  Inputs,  and  a 
low  rate  telemetry  output. 


Figure  2  is  a  functional  representation  of 

the  spun  and  despun  sections  of  the  CDS. 

A  brief  discussion  of  the  Illustrated  ele¬ 
ments  follows: 

1 .  Hardware  Command  Decoder  (HCD) .  This 
is  the  basic  communication  path  to  the 
CDS  and  the  spacecraft  from  the 
ground.  A  serial  bit  stream  is  de¬ 
coded  and  routed  to  the  designated 
CDS  element  or  Orbiter  subsystem. 

2.  Critical  Controller  (CRC).  There  is 
a  redundant  pair  in  both  CDS  sections 
serving  as  the  "lock  and  key"  to  protect 
critical  functions,  random  writing  in 
memory,  and  establish  the  configura¬ 
tion  of  the  CDS.  It  may  be  accessed 
by  the  Hardware  Command  Decoder  or  by 
the  CDS  High  Level  Modules  under 
special  conditions. 

3.  High  Level  Modules  (HLM) .  Each  con¬ 
tains  an  RCA  1802  microprocessor^ 
operated  at  a  clock  frequency  of 
1.6128  mHz  with  32K  Read  Write  Memory 
(RWM)  consisting  of  eight  bit  words 
with  error  detection  parity.  The  32K 
RWM  is  "write-protected"  in  4K  blocks. 
In  addition,  each  High  Level  Module 
contains  a  Bus  Adaptor  (BA)  which 
shares  a  complete  "off-chip"  Direct 
Memory  Access  (DMA)  function  allowing 
the  bus  system  to  enter  or  extract 
data  from  the  memory.  Each  High  Level 
Module  also  has  a  bus  controller 
which,  like  the  BA,  is  dedicated  to 
its  bus.  The  BC,  which  is  unique  to 
High  Level  Modules,  can  move  blocks 

of  data  between  memories  of  all 
modules,  connected  to  its  bus.  The 
bus  controller,  operating  independent¬ 
ly  from  the  microprocessor,  reads  a 
control  table  directly  from  the  mem¬ 
ory  to  specify  the  bus  transfers. 

4.  Low  Level  Modules  (LLM) .  Each  con¬ 
tains  an  RCA  1802  microprocessor 
operated  at  a  clock  frequency  of 
1.6128  mHz  and  16K  RWM.  In  addition, 
each  Low  Level  Module  contains  a  BA. 


5.  Bulk  Memories  (BUM) .  Each  contains 
16K  RWM  and  two  BAs  which  can  indi¬ 
vidually  be  assigned  to  any  bus  by  the 
critical  controllers.  Also,  each 
Bulk  Memory  contains  a  telemetry  out¬ 
put  and  format  interface  which  can  be 
outputted  to  the  downlink  at  rates  up 
to  115.2  kbps. 

6.  Data  Memory  Bulk  Memories  (DBUM) . 

Each  contains  8K  RWM  and  a  BA  which 
can  be  assigned  to  any  bus.  Addi¬ 
tionally,  there  are  Interfaces  for 
inputting  high  rate  data  and  output¬ 
ting  and  formatting  tape  recorder 
high  rate  data. 

7.  Golay  Coders  (GC).  Provide  the  capa¬ 
bility  to  encode  432  bits  of  data 
using  Golay  (24,12)  code  with  an  in¬ 
terleaved  depth  of  36.  The  (k)lay 
Coder  BA  can  be  assigned  to  either 
bus. 

8.  Multiplexers  (MUX) .  Under  the  control 
of  the  bus  systems,  a  reply  data  in¬ 
put  (R)  is  selected  from  the  internal 
and  external  bus  users  and  routed  to 
the  Command  and  Data  "Supervisory" 
bus. 

9.  Timing  Chains  (TC) .  Provide  the  basic 
timing  internal  to  the  CDS  and  to  the 
external  bus  users.  The  basic 
Orbiter  time  is  the  distributed  Real- 
Time-Interrupt  which  is  66-2/3  milli¬ 
seconds.  Elements  Internal  to  the 
CDS  are  given  the  1.6128  mHz  clock  and 
RTI.  External  users  are  given  RTI 
and  806.4  kHz  from  which  they  derive 
bit  and  word  sync  for  bus  communica¬ 
tion.  Distribution  of  this  timing  is 
phase  lock  looped  to  insure  operation 
through  transients. 

10.  Command  and  Data  Bus  (CDB).  Contains 
three  (3)  dedicated  busses.  Two  are 
individually  controlled  by  the  dedi¬ 
cated  BC  of  a  High  Level  Module.  The 
third  is  controlled  by  the  CDS  support 


equipment  during  Initial  memory  load¬ 
ing  and  not  used  otherwise. 

The  purpose  of  the  CDB  Is  to  trans¬ 
fer  data  from  one  memory  to  another. 
All  commanding,  memory  loading,  and 
data  collecting  are  Implemented 
through  this  mechanism. 


A  typical  data  path  consists  of  the 
bus  controller  which  manages  the  bus 
transaction,  the  single  bus  adaptor 
which  Is  selected  as  a  data  source, 
and  the  one  or  more  bus  adaptors  which 
are  selected  as  data  recipients.  Bus 
controller  Instructions  are  passed 
over  the  "supervisory"  bus  at  a  403.2 
kbps  rate  to  the  bus  adaptors.  The 
reply  data  Is  passed  over  a  reply 
line  after  having  been  selected  as 
one  signal  path  out  of  many  by  the 
multiplexer.  This  allows  the  control¬ 
ler  to  retain  management  of  traffic 
on  the  supervisory  data  bus  which  is 
the  data  distribution  path. 


Software  Description 

The  CDS  software  functions  are  distri¬ 
buted  amongst  the  various  elements  as  In¬ 
dicated  In  Figure  4.  Although  the  HLMs 
and  LLMs  are  the  only  elements  which  con¬ 
tain  a  processor,  the  BUMs  and  DBUMs  are 
Included  because  they  contain  memory 
which  Is  managed  as  software.  The  soft¬ 
ware  elements  can  be  described  briefly  as 
follows: 

1.  High  Level  Modules  (HLM)  -  The  HLM 
serves  as  the  only  direct  software 
recipient  of  command  data.  Some  of 
that  data  Is  meant  for  the  HLM  and  Is 
either  decoded  as  a  processor  com¬ 
mand  and  executed,  moved  as  memory 
or  sequence  storage  data,  or  temporar¬ 
ily  stored  in  memory  to  be  interpret¬ 
ed  as  command  data  at  a  future, 
specified  time.  Data  which  Is  not 
meant  for  the  HLM  can  be  memory  data 
or  commands  which  are  moved  to  the 
appropriate  elements  either  internal 
or  external  to  the  CDS  via  the 
Command  and  Data  Bus. 


11.  Power  Converters  (PC).  The  redundant 
power  converters  can  each  supply  power 
for  the  entire  CDS.  As  Indicated,  the 
CDS  despun  section  power  is  supplied 
from  the  CDS  spun  section  power  con¬ 
verters  through  the  spin  bearing 
assembly. 

The  CDS  flight  configuration  is  shown  in 
Figure  3.  The  Command  and  Data  Bus  con¬ 
trolled  by  the  HLM  lA  is  considered  the 
prime  data  bus  while  the  bus  controlled 
by  HLM  IB  is  the  back-up.  The  prime  data 
bus  services  the  interfaces  with  all  ex¬ 
ternal  bus  users  and  is  also  used  to  col¬ 
lect  and  format  the  high  data  rate  tele¬ 
metry.  During  noncfltlcal  mission  phases 
the  HLM  lA  bus  will  be  generally  used  in 
a  single  string  mode  of  operation  with  the 
HLM  IB  bus  serving  as  back-up.  The  HLM  IB 
bus  will  serve  an  active  role  during  all 
critical  mission  phases  by  issuing  the 
mission  critical  commands  through  Its  own 
set  of  redundant  Interfaces  (mainly 
through  LLM  IB  and  LIM  2B).  In  the  event 
of  a  failure  the  two  data  bus  roles  can  be 
Interchanged,  with  the  HLM  IB  bus  serving 
as  the  prime  data  bus. 


The  Command  and  Data  Bus  Is  also  used 
to  distribute  the  spacecraft  clock 
which  is  maintained  by  the  HLM  soft¬ 
ware  and  used  to  coordinate  all 
spacecraft  activities.  Besides  serv¬ 
ing  as  the  driver  for  the  temporarily 
stored  command  data,  the  spacecraft 
clock  is  also  used  by  the  HLM  soft¬ 
ware  for  the  execution  of  stored  se¬ 
quences.  That  data  Is  in  the  form  of 
time-event  tables  which  are  time- 
ordered  according  to  an  absolute  or 
relative  clock.  At  the  appropriate 
times  the  events  are  interpreted  as 
command  data  and  the  corresponding 
actions  taken.  Besides  the  simple 
actions  previous  described  and  called 
events,  more  complex  actions,  called 
pseudo-events  can  be  taken.  These 
pseudo-events  are  parameterized  rou¬ 
tine  calls  which  Initiate  algorithms 
causing  subsequent  activities.  Fre¬ 
quently  these  activities  result  in 
the  creation  of  a  large  number  of  com 
mands  and  the  process  has  become 
known  as  sequence  expansion. 

Besides  the  uplink  commands  and 
stored  sequences,  another  source  of 
spacecraft  commands  is  the  HIM  alarm 
monitoring  function.  This  function, 
controlled  by  the  HLM  software. 


monitors  status  and  performance  data 
which  Is  either  directly  available  to 
the  HLM  or  Is  made  available  through 
the  command  and  data  bus.  That  data 
Is  compared  against  anticipated  per¬ 
formance  and  spacecraft  states  and 
any  deviations  result  in  the  initia¬ 
tion  of  an  error  recovery  process, 
frequently  consisting  of  activating 
a  previously  stored  time-event  table 
which  generates  corrective  commands. 

As  can  be  seen,  the  HLM  software  Is 
the  source  of  all  spacecraft  commands 
which  are  either  executed  internally 
by  the  HLM  or  distributed  to  the 
spacecraft  via  the  command  and  data 
bus.  This  bus,  which  Is  controlled 
by  the  HLM  software.  Is  also  used  to 
control  all  data  movement  on  board  the 
spacecraft.  That  data  is  used  for 
alarm  monitoring,  downlink  or  record 
telemetry,  or  memory  readout.  Alarm 
monitor  data  must  be  moved  to  the  HLK 
for  processing.  Downlink  telemetry 
and  memory  readout  data  must  be  moved 
to  the  BUM  for  formatting  while  the 
same  data  must  be  moved  to  the  DBUM  to 
be  formatted  for  recording. 

2.  Low  Level  Modules  (LLM)  -  The  LLM  soft¬ 
ware  is  subservient  to  the  HLM  soft¬ 
ware  and  Interfaces  with  It  through 
the  command  and  data  bus.  Although 
the  LLM  maintains  time  for  synchroni¬ 
zation,  it  will  resync  if  there  is  a 
difference  between  Its  time  and  the 
time  distributed  by  the  HLM.  All  LLM 
commands  are  received  from  the  HLM  and 
either  executed  internally  as  LLM  proc¬ 
essor  commands  or  queued  and  subse¬ 
quently  Issued  to  other  subsystems  as 
discrete  (relay  closure)  or  digital 
(serial  binary  data  stream)  commands. 
The  LLM  software  Is  also  responsible 
for  the  collection  of  status  Informa¬ 
tion  from  Itself  and  those  subsystems 
which  Interface  directly  with  the  LLM 
rather  than  with  the  command  and  data 
bus.  Although  the  status  collection  Is 
table  driven  It  represents  a  large  task 
for  the  LLM  because  the  data  must  be 
subcommutated  and  properly  "packetlzed" 
for  collection  by  the  HLM  bus.  The  HLM 
bus  also  routes  the  low  rate  telemetry 
to  the  LLM  where  It  Is  buffered  prior 
to  being  sent  down  to  the  ground. 

3.  Bulk  Memories  (BUM)  -  The  BUM  "soft¬ 
ware"  Interfaces  with  the  other 


software  elements  via  the  HLM  bus. 

Its  memory  capacity  is  used  to  store 
error  recovery  algorithms  not  fre¬ 
quently  required  by  the  HLM  and 
stored  sequences.  Its  memory  also 
contains  buffers  for  telemetry  In¬ 
formation  which  is  formatted  and  out¬ 
put  over  the  high  rate  downlink  tele¬ 
metry  channel,  and  buffers  which  are 
used  for  Interbus  communication. 

4 .  Data  Memory  Bulk  Memories  (DBUM)  - 
The  DBUM  "software"  also  interfaces 
with  the  other  software  elements  via 
the  HLM  bus.  Its  memory  contains 
buffers  for  telemetry  information 
which  is  formatted  and  output  to  the 
tape  recorder  (Data  Memory  Subsystem) 
and  buffers  which  are  used  for  the 
high  rate  solid  state  imaging  (SSI) , 
plasma  wave  subsystem  (PUS) ,  and 
playback  input  data. 

The  basic  microprocessor  software  struc¬ 
ture  (HLM  and  LLM)  is  shown  in  Figure  5. 
The  foreground  executive  is  driven  by  the 
hardware's  real  time  interrupt  (RTI) 
which  occurs  once  each  66-2/3  msec.  This 
exec  handles  timekeeping  and  the  hardware 
software  coordination  which  is  done  at 
the  RTI  level.  It  also  passes  control  to 
the  background  executive  which  works  on  a 
ten  RTI  basis  and  is  responsible  for  all 
the  other  software  functions  which  are 
summarized  in  the  figure. 

Hardware/Software  Interface 

The  hardware  and-  software  interface 
occurs,  in  time,  at  the  RTI  level.  The 
hardware  is  responsible  for  all  timing 
finer  than  an  RTI  while  the  software  is 
responsible  for  all  timing  coarser  than 
an  RTI.  Hardware  activities,  established 
by  the  software  during  an  RTI  interval, 
are  activated  on  the  next  RTI.  This 
keeps  the  hardware-software  synchroniza¬ 
tion  at  an  RTI  level  and  eliminates  any 
fine  timing  interactions.  This  principle 
Is  illustrated  in  Figure  6.  The  software 
functions  start  with  the  RTI  driven  space¬ 
craft  clock  followed  by  the  10  RTI  driven 
functions.  Some  of  the  hardware  func¬ 
tions,  all  driven  by  RTI,  are  also 
illustrated. 

The  hardware  is  controlled  by  the  software 
through  writing  in  out-of-range  memory 
addresses  (memory  mapped  I/O)  and  through 
direct  memory  accessed  (DMA)  tables  when 
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timing  finer  than  an  RTI  is  required. 

The  software  attains  status  information 
from  the  hardware  through  reading  out-of¬ 
range  memory  addresses  (also  memory 
mapped  I/O)  and  from  DMA  tables  when 
timing  finer  than  an  RTI  is  required. 

To  illustrate  these  points  the  six  exam¬ 
ples  of  Figure  6  are  discussed. 

1.  Bus  Control 

a)  The  address  of  the  bus  control 
table  is  written  into  an  out-of¬ 
range  address. 

b)  At  the  next  RTI  the  table  is  read 
out  of  memory  by  the  bus  control¬ 
ler  and  used  to  transfer  data  be¬ 
tween  memories  connected  to  the 
bus.  The  reading  of  the  bus  con¬ 
trol  table,  the  reading  of  mem¬ 
ory,  and  the  writing  of  memory 
are  all  through  DMA. 

2.  Status/Telemetry 

a)  The  address  of  the  channel  iden¬ 
tifier  table  (and  consequently 
the  input  data  table)  is  written 
into  an  out-of-range  address. 

b)  At  the  next  RTI  the  words  in  the 
channel  identifier  table  are  indi¬ 
vidually  read  from  memory,  the 
identified  channel  is  sampled, 
and  the  input  data  is  individual¬ 
ly  written  into  memory.  The  read 
and  write  memory  processes  are 
both  DMA. 

3.  Telemetry  Out 

a)  The  address  of  the  telemetry  for¬ 
matter  table  is  written  into  an 
out-of-range  address. 

b)  At  the  next  RTI  four  words  of  the 
telemetry  formatter  table  are 
read  out  of  memory  to  establish 
the  address  of  the  first  data 
block  and  the  number  of  words  it 
contains. 

c)  On  the  same  RTI,  immediately 
after  b) ,  the  first  data  block  is 
read  out  and  sent  down  telemetry 
by  the  telemetry  sequencer. 

d)  At  the  conclusion  of  c)  the  next 
four  words  of  the  telemetry 


formatter  are  read  out  of  memory 
to  establish  the  address  of  the 
next  data  block  and  the  number 
of  words  it  contains. 

e)  Immediately  after  d) ,  the  next 
data  block  is  read  out  and  sent 
down  telemetry  by  the  telemetry 
sequencer. 

f)  Items  d)  and  e)  are  repeated  in¬ 
definitely  until  reinitialized 
by  item  a) . 

4.  Record  Data  Out 

a)  Fundamentally  this  is  the  same 
process  as  Telemetry  Out. 

5.  Hi  Rate  Data  In 

a)  The  address  of  the  Hi  Rate  Data 
In  buffer  is  written  into  an 
out-of-range  address. 

b)  At  the  next  RTI  subsequent  in¬ 
coming  data  words  are  stored 
consecutively  in  the  Hi  Rate 
Data  buffer  starting  at  the  be¬ 
ginning  address. 

6.  Dlgltal/Dlscrete  Commands 

a)  The  two  bytes  of  command  data 
(digital  or  discrete)  are  written 
into  an  out-of-range  address. 

b)  At  the  next  RTI  the  command  ex¬ 
ecution  process  is  initiated. 

(It  requires  2  RTls  for 
completion. ) 

Command  and  Data  Bus 

The  heart  of  the  spacecraft  communications 
is  the  command  and  data  bus.  This  bus 
serves  as  the  major  link  between  the  vari¬ 
ous  elements  of  the  CDS,  the  CDS  and  the 
majority  of  the  spacecraft  subsystems,  and 
the  spun  and  despun  sections  of  the  space¬ 
craft.  Subsystems  which  do  not  interface 
directly  with  the  bus,  interface  with  the 
low  level  modules  which  make  the  subsystem 
unique  Interfaces  compatible  with  the  bus. 

The  format  of  the  bus  is  shown  in  Figure  7. 
The  header  portion  contains  control  in¬ 
formation  consisting  of  groups  of  3  words 
followed  by  a  final  word.  Each  3-word 
group  identifies  a  user  ,  whether  the  user 
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is  to  be  a  source  or  recipient,  and  a  16- 
bit  address  in  memory  for  the  beginning 
of  the  data  transfer.  Although  multiple 
recipients  are  allowed,  only  one  source 
is  permitted.  The  final  word,  called  the 
number  -  of-words  word,  tells  the  bus  con¬ 
troller  how  many  data  bytes  are  to  be 
transferred  and  informs  the  participants 
that  the  transfer  is  about  to  begin.  The 
transfer  consists  of  O-to-127  words  of 
data  (sometimes  the  address  Information 
is  the  message)  and  it  is  terminated  by 
the  bus  controller  smarting  another  bus 
transaction.  In  those  cases  where  there 
are  no  more  transactions  scheduled  the 
bus  controller  puts  out  a  filler  pattern 
which  also  serves  to  terminate  the  preced¬ 
ing  transaction.  The  filler  pattern  con¬ 
tinues  until  more  transactions  are  init¬ 
iated  at  the  next  RTI.  These  transac¬ 
tions  are  in  reality  memory-to-memory 
transfers  used  to  distribute  and  collect 
data.  The  distributed  data  is  typically 
command  or  memory  load  Information  while 
the  collected  data  is  usually  telemetry, 
monitor,  or  memory  readout  data. 

Recognizing  that  the  bus  is  really  a  com¬ 
munication  path  between  distributed  memo¬ 
ries  and  that  most  of  these  memories  have 
multiple  DMAs,  one  can  envision  the  CDS 
and  its  external  bus  users  as  an  inter¬ 
connection  of  multiport  memories.  An  ex¬ 
ample  of  this  concept  is  shown  in  Figure 
8.  While  this  figure  does  not  represent 
all  of  the  external  bus  users  or  the  CDS 
it  does  give  insight  into  the  basic  con¬ 
cept.  Allocating  the  basic  functions 
described  in  the  hardware/sof tware  inter¬ 
face  section  to  Figure  8  can  be  done  as 
follows : 

1.  Bus  Control 

a)  The  Bus  Controller  is  shown  as 
the  right  block  in  the  High  Level 
Module . 

b)  The  bus  adapters  are  shown  as  the 
bottom  blocks  in  the  top  elements 
and  the  top  blocks  In  the  bottom 
elements. 

c)  The  Bulk  Memory  has  a  second  bus 
adapter  shown  as  its  right  block 
which  Interfaces  with  the  other 
command  and  data  bus. 


2.  Status/Telemetry  In 

a)  This  is  the  bottom  block  on  the 
right  in  the  Low  Level  Module. 

3.  Telemetry  Data  Out 

a)  The  telemetry  formatter  is  shown 
as  the  left  block  in  the  Bulk 
Memory. 

b)  The  telemetry  sequencer  is  shown 
as  the  bottom  block  in  the  Bulk 
Memory . 

4.  Record  Data  Out 

a)  This  is  similar  to  the  Telemetry 
Data  Out  except  the  blocks  reside 
in  the  Data  Memory  Bulk  Memory. 

5.  Hi  Rate  Data  In 

a)  This  is  the  right  block  in  the 
Data  Memory  Bulk  Memory. 

6.  Digltal/Dlscrete  Commands 

a)  This  is  the  bottom  block  on  the 
left  side  in  the  Low  Level  Module. 

A  few  examples  help  illustrate  the  con¬ 
cept.  Uplink  data  arrives  at  the  Hard¬ 
ware  Command  Decoder  where  it  is  initially 
decoded  and  passed  on  to  the  High  Level 
Module  through  the  command  interface  DMA. 
Periodically  the  HLM  1802  microprocessor 
checks  for  the  presence  of  command  data 
in  order  to  further  decode  it.  If  that 
data  is  not  for  the  HLM  it  schedules  a 
future  bus  transaction  to  move  it  to  the 
appropriate  recipient.  If  the  data  was  a 
bus  command  the  activity  would  be  conclud¬ 
ed  with  the  bus  transaction.  If,  however, 
the  data  was  a  digital  or  discrete  com¬ 
mand  it  would  be  passed  to  the  Low  Level 
Module.  The  LLM  1802  microprocessor 
which  periodically  checks  for  the  presence 
of  command  data  would  find  the  digital  or 
discrete  command  and  place  it  in  a  queue 
for  execution.  As  the  LLM  1802  micropro¬ 
cessor  advanced  the  queue  it  would  event¬ 
ually  get  to  the  subject  discrete  or  digi¬ 
tal  command  and  place  it  in  the  out-of- 
range  memory  addresses  for  activation.  On 
the  next  RTI  the  hardware  would  issue  the 
relay  closure  or  send  the  binary  serial 
data  to  the  appropriate  user  and  the  ac¬ 
tivity  would  be  concluded . 
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Another  example  would  be  the  processing 
of  engineering  telemetry  status.  This 
data  is  available  through  interfaces 
with  the  Low  Level  Module.  The  request 
for  that  data  plus  others  is  loaded  into 
a  table  according  to  a  previously  speci¬ 
fied  schedule.  The  engineering  status 
hardware  reads  the  request,  samples  the 
data,  and  stores  the  result  into  the 
data  table.  The  LLM  1802  will  then  take 
the  information  from  the  data  tables  and 
"packet ize"  it  into  concurrent  memory 
locations  for  pickup  by  the  bus.  During 
the  specified  RTI,  that  data  is  moved 
from  the  LLM  and  other  engineering  tele¬ 
metry  sources  to  build  a  larger  telemetry 
collection  in  the  Bulk  Memory.  From 
there  that  information  is  picked  up  by 
the  telemetry  sequencer  under  directions 
from  the  telemetry  formatter  and  the  data 
is  sent  down  the  downlink. 


Summary  of  CDS  Characteristics 


1.  Distributed  architecture  based  on 
high-level  and  low-level  modules. 

The  core  of  these  modules  is  the  RCA 
1802  microprocessor  and  the  high 
capacity  bus  adaptors  and  control¬ 
lers.  1802  characteristics  are: 


a)  Radiation-hardened  CMOS 
technology 

b)  95  op  codes  (register  oriented) 

c)  Operated  at  1.6128  mHz  yielding 
approximately  5  microseconds  per 
cycle.  Most  computational  opera¬ 
tions  are  two  machine  cycles. 

d)  DMA  capability 

2.  Utilizes  CMOS  technology  throughout 

3.  Planar  (hlgh-denslty)  packaging  on 
multilayer  boards  mounted  on  honey¬ 
comb  structure. 


4.  Estimated  power:  30  watts 

5.  Estimated  weight:  30  Kgm 

3 

6.  Estimated  volume:  3000  in 


7.  Six  RCA  1802  microprocessors. 

8.  Software  basically  table  driven 
under  the  control  of  executive 
structure. 


9.  176K  bytes  of  storage  in  10  memories 

10.  Supervisory  Bus  Rate:  403.2  kbps 

11.  Downlink  Telemetry:  115.2  kbps 
maximum 

12.  RTI:  66-2/3  milliseconds 

13.  Maximum  Record  Data  Rate:  806.4  kbps 

This  paper  presents  the  results  of  one 
phase  of  research  performed  at  the  Jet 
Propulsion  Laboratory,  California, 
California  Institute  of  Technology,  spon¬ 
sored  by  the  National  Aeronautics  & 

Space  Administration  under  Contract 
NAS  7-100. 
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CDS  H.IOHT  CONFieURATION 


HARDWARE/SOFTWARI  INFRACTION 
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ABSTRACT 

BIT  SLICE  MICROPROCESSOR  INFLUENCE  ON  A 
SPACE80RNE  HIGH  THROUGHPUT  PROCESSOR 
ARCHITECTURE 

L.F.  Bonilla  -  Hughes  Aircraft  Co. 

K.B.  Smernoff  -  Hughes  Aircraft  Co. 

D.D.  Vesely  -  Hughes  Aircraft  Co. 

The  High  throughput  Processor  is  an 
advanced  central  processing  unit  designed 
for  real  time  space  processing  systems. 

The  primary  requirements  imposed  on  the 
processor  were  1)  radiation  hardening  for 
both  total  dose  and  prompt  dose  protec¬ 
tion,  2)  minimum  power,  3)  high  reliabil¬ 
ity  for  a  10  year  mission,  4)  high 
throughput  capability.  Development  risk 
was  to  be  a  minimum. 

The  architecture  selected  for  use  in  the 
processor  design  was  the  result  of  trade¬ 
off  studies  that  examined  various  micro¬ 
processor  architectures  and  device  tech¬ 
nologies.  The  architecture  and  device 
technology  that  best  met  the  space  design 
requirements  uses  the  Advanced  Micro 
Devices  2900  series  low  power  Schottky 
(TTL)  integrated  circuits.  The  primary 
element  in  the  processor  architecture  is 
the  2901  4-bit  microprocessor  slice  which, 
along  with  other  2900  series  parts,  forms 
a  16-bit  parallel  processor. 

The  processor  is  a  single  phase  synchro¬ 
nously  clocked  unit  with  the  machine 
instruction  set  implemented  through  a 
microprogrammed  control  unit.  The 
instruction  set  is  a  derivative  of  the 
Hughes  F-18  radar  data  processor  instruc¬ 
tion  set.  This  instruction  set  was 
selected  primarily  because  it  is  proven 
and  is  compatible  with  the  2901  micro¬ 
processor  architecture,' 

The  High  Throughput  Processor  design  con¬ 
sists  of  the  arithmetic  processing  unit, 
timing  and  control  logic,  interrupt  logic, 
power  up  and  power  down  logic,  and  system 
test  equipment  (STE)  interface  logic.  The 
power  up  and  power  down  logic  allows  the 
HTP  to  be  power  strobed  to  minimize  power 
as  a  function  of  throughput.  The  STE 
allows  serial  access  to  the  processor 
registers  at  the  system  checkout  level. 

The  support  software  developed  for  the  HTP 
consists  of  a  cross-assembler,  instruction 


set  simulator  (non-real-time)  and  a  diag¬ 
nostics  package. 

The  breadboard  design  and  checkout  of  the 
HTP  has  been  completed,  and  the  HTP  space 
quality  printed  circuit  boards  are  being 
designed.  The  HTP  will  be  qualified  and 
then  delivered  to  a  spacecraft  in  the 
fourth  quarter  of  1981.  A  major  portion 
of  this  work  was  performed  under  USAF 
Contract  F04701-77-C-0100  and 
F04701-79-C-0006. 

BIT  SLICE  MICROPROCESSOR  INFLUENCE 
ON  SPACEBORNE,  HIGH  THROUGHPUT 
PROCESSOR  ARCHITECTURE 


HIGH  THROUGHPUT  PROCESSOR  (HTP) 
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•  REAL  TIME  COMTROl 

•  OATAANO  COMMUNICATIONS  PROCESSING 

•  SPACECRAFT  MANAGEMENT 

OPTIONAL  PORiE  R  STROBING  TO  MINIMIZE  POWER  AT 
REDUCED  PROCESSING  LOADS 


HIGH  THROUGHPUT  PROCESSOR  (HTP) 

KEY  FEATURES 

DEVICE  TECHNOLOGY 

•  CPU  LOGIC  LOW  POWER  SCHOTTKY  TTL  (2900  SERIES  OCVICESI 

•  PROM  CONTROL  MEMORY:  SCHOTTKY  TTL.  NfCHROME  FUSED  LINK 
RADIATION  TOLERANCE 

•  >  10®  RAO  (SiJ  TOTAL  DOSE 

•  PROMPT  DOSE  PROTECTION 
ACCESSORY  CIRCUITS  INCLUDED  WITH  CPU 

•  COMPLETE  CLOCK  TIMING  AND  CONTROL  LOGIC 

•  SPECIAL  TEST  EQUIPMENT  INTERFACE  TO  EASE  PROGRAM  CHECKOUT  AND 
SYSTEM  INTEGRATION 

POWER 

•  WITHOUT  POWER  STROBING  18  W 

•  WITH  POWER  STROBING  20  WAT  MAX  PROCESSING  SPEED  (POWER  UPl 

0.5  W  STANDBY  (POWER  DOWN) 

AVERAGE  POWER  VS  DUTY  CYCLE.  LINEAR 


HIGH  THROUGHPUT  PROCESSOR  (HTP) 
FUNCTIONAL  ARCHITECTURE 
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HIGH  THROUGHPUT  PROCESSOR  (HTP) 
FUNCTIONAL  CHARACTERISTICS 

•  WORD  LENGTH:  16  BITS 

•  DATA  LENGTHS:  32,  16,  8,  4, 1  BITS 

•  INSTRUCTIONS:  114 

•  ARITHMETIC:  FRACTIONAL,  2'S  COMPLEMENT 


2900  SERIES  LSI 
VS  MSI/SSI 
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7  LSI  COMPONENTS  EQUIVALENT  TO  -  95  MSI/SSI 

LESS  PARTS  FOR  IMPROVED  RELIABILITY  AND  SMALLER  SIZE 


•  USER  REGISTERS:  10  (8  GENERAL  PURPOSE) 

•  INTERRUPTS:  8  (1  POWER  STROBE) 

•  I/O:  MEMORY  MAPPED 

•  MEMORY  ADDRESSING:  15  MODES, 

UP  TO  64K  WORDS 


HTP  SUPPORT  RESOURCES 

SUPPORT  EQUIPMENT 

•  CPU  TESTER 

•  CRT/KEYBOARD  MONITOR  AND  CONTROL 
SUPPORT  SOFTWARE 

•  CROSSASSEMBLER 


HIGH  THROUGHPUT  PROCESSOR  (HTP) 
FUNCTIONAL  SPEED  CHARACTERISTICS 
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•  SIMULATOR 

•  CPU  DIAGNOSTICS  (RESIDENT  IN  HTP) 

•  HIGHER  ORDER  LANGUAGE  (HOD  COMPILER  (PLANNED) 

CONCLUSIONS 

2900  SERIES  BIPOLAR  BIT  SLICE  FAMILY  AND  BIPOLAR  PROM'S  MEET 
HIGH  PERFORMANCE  NEEDS 

•  VERSATILE  BUILDING  BLOCKS  WITH  GOOD  FLEXIBILITY  FOR 
SPACEBORNE  PROCESSORS 

•  WILL  SATISFY  MANY  SPACECRAFT  NEEDS  FOR  NEXT  6  TO  10 
YEARS  OR  MORE 

•  GOOD  SPEED  AND  RADIATION  TOLERANCE  CHARACTERISTICS 


•  RELATIVELY  HIGH  POWER,  CAN  BE  REDUCED  VIA  POWER  STROBING 
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RECOMMENDATIONS  FOR 
BIT  SLICE  AND  MEMORY  DEVICES 

UPGRADE  HIGH  PERFORMANCE  BIPOLAR  LSI  COMPONENTS  BY 
ADDING  POWER  STROBE  FEATURES 

•  ON-CHIP  POWER  SWITCH  PROVISIONS  (BIT  SLICE  FAMILY  AND 
PROM'S) 

ENCOURAGE  DEVELOPMENT  OF  LOW  POWER.  RAD  HARD  FAMILY  OF 
HIGH  SPEED  DEVICES 

•  CMOS  OR  COMS-SOS  BIT  SLICE  FAMILY  -  8  BITS  WIDE 

•  GATE  ARRAYS  FOR  PERIPHERAL-LOGIC 

•  RAM'S;  UP  TO  4K  X  1  OR  LARGER 

•  PROM'S:  UP  TO  4K  X  8  OR  LARGER 


CMOS/SOS  HIGH-PERFORMANCE 
PROCESSORS  FOR  FUTURE 
SPACEBORNE  MISSIONS 

A.  F.  Cornish,  S.  E.  Ozga,  J.  E.  Saultz 
RCA,  Advanced  Technology  Laboratories 
Camden,  New  Jersey 

ABSTRACT 

The  successful  aiipllcatlon  of  microprocessor- 
based  systems  to  space  applications  in  the 
1980 's  and  90 's  presents  a  challenge  to  space 
system  designers,  computer  architectural 
designers  (hardware/software),  semiconductor 
technologists  and  advanced  packaglng/manu- 
factuiing  organizations.  Valuable  experience 
from  the  1960's  and  70's  in  developing  space 
systems  has  shown  that  the  approach  of 
building  a  processor  and  then  applying  that 
processor  to  a  broad  set  of  requirements  was 
one  of  constant  compromise  between  cost, 
weight,  power,  reliability  throughput  and 
radiation  tolerance.  The  successful  use  of 
microprocessors  in  space  systems  requires  a 
thorough  knowledge  of  space  applications  as 
well  as  an  awareness  of  new  or  anticipated 
technology  developments  that  may  be  appli¬ 
cable. 

RCA  has  embarked  on  a  program  which  com¬ 
bines  its  space  experience  with  CMOS/SOS 
technology  to  develop  a  family  of  processors 
for  space  applications.  The  CMOS/SOS 
technology  is  generally  recognized  as  a  VLSI 
technology  with  numerous  advantages  for 
military  and  aerospace  applications.  RCA  is 
capitalizing  on  this  technology  by  applying  it  in 
tigldly  constrained  military  and  commercial 
systems.  It  is  providing  attractive  solutions 
to  problems  that  cannot  be  matched  by  the 
more  prevalent  commercial  technologies. 
Specifically,  we  have  developed  VLSI  micro¬ 
processors  and  siqjport  circuits  for  space 
applications  which  provide  high-speed  signal 
processing,  low  power  and  environmental 
tolerance.  This  paper  describes  some  of  the 
technology  advantages  and  advanced  packaging 
approaches  of  CMOS/SOS,  and  presents  a 
more  detailed  discussion  of  results  currently 
being  achieved  with  our  htgh-performance  data 
and  signal  processors.  In  particular,  charac¬ 
teristics,  results  and  applications  of  the  RCA 


ATMAC  microprocessor  and  the  Air  Force 
sponsored  GP  U  chip  set  will  be  presented. 

The  tremendous  potential  of  CMOS/SOS  has 
been  demonstrated  in  numerous  RCA  processor 
designs  and  embedded  processor  applications. 
One  of  our  most  successful  ventures  is  the 
SCP234  spacebome  computer  which  is  cur¬ 
rently  in  service  In  the  TIROS  and  DMSP 
satellites.  This  computer  employs  a  CMOS/ 
SOS  memory  unit  and  has  provided  us  with 
significant  reliability  data  on  the  technology. 
The  ATMAC  microprocessor,  developed  by 
RCA  Advanced  Technology  Laboratories,  is 
an  excellent  example  of  a  signal  processing 
system  with  a  throughput  potential  of  more 
than  8  million  operations  per  second  while 
dissipating  less  than  3/4  W  of  dynamic  power. 
AT  mac's  potential  is  being  exploited  in 
numerous  computational  intensive  applications 
having  severe  size,  weight  and  power  con¬ 
straints.  For  example,  Hughes  Aircraft  Co. 
has  selected  a  multiprocessor  ATMAC  con¬ 
figuration  for  the  target-tracking  functions  in 
the  Mlnl-HALO  Experiment,  a  DARPA 
sponsored  mosaic-sensor,  spacebome  sur¬ 
veillance  system  targeted  for  launch  in  the 
early  1980’s. 

For  lower  performance  applications,  the  GPU 
chip  set  developed  by  the  Air  Force  provides 
an  excellent  technology  implementation  which 
is  capable  of  emulating  commercial  and  mili¬ 
tary  general-purpose  computers.  This  capa¬ 
bility  is  becoming  increasingly  attractive  as 
DoD  and  NASA  intensify  their  efforts  toward 
general-purpose  computer  standardization  at 
the  Instructioneset  architecture  level. 

INTRODUCTION 

The  increasing  sophistication  of  aerospace 
applications  demands  that  computers  of  the 
1980 's  and  1990*8  be  capable  of  processing 
Increasingly  large  volumes  of  data,  much  of 
it  in  real  time.  These  demands  present  the 
system  designer  with  an  almost  insurmountable 
challenge,  that  of  designing  the  optimal  proces¬ 
sor  configuration  for  his  particular  application 
within  the  constraints  of  weight,  power, 
reliability  and  cost.  Fortunately,  significant 
advances  have  been  occurring  in  VLSI  cir¬ 
cuit  technology,  packaging  and  computer  aided 
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design.  These  provide  the  designer  with  tools 
that  greatly  simplify  his  task.  VLSI  technol¬ 
ogy  Is  making  available  reliable  high-speed 
components,  while  new  packaging  techniques 
are  providing  dense  and  compact  electronic 
assemblies.  Computer  aided  design  techniques 
applied  to  both  areas  are  significantly  reducing 
design  time  and  non-recurring  development 
costs. 

RCA  is  using  CMOS/SOS  technology  in  both 
standard  and  radiation  tolerant  forms  as  the 
basis  for  developing  a  family  of  militarized 
data  and  signal  processors.  These  proces¬ 
sors  are  designed  for  embedded  applications 
where  the  more  prevalent  commercial 
technologies  cannot  provide  the  required  levels 
of  performance. 

This  paper  briefly  describes  some  of  the 
technology  advantages  and  advanced  packaging 
approaches  of  CMOS/SOS  and  a  more  de¬ 
tailed  discussion  of  results  currently  being 
achieved  in  one  of  its  highest  payoff  applica¬ 
tions  —  high-performance  data  and  signal 
processors.  In  particular,  characteristics, 
results  and  applications  of  the  RCA  ATMAC 
and  the  Air  Force  sponsored  GPU  chip  set 
will  be  presented, 

CMOS/SOS  Technology 

Of  the  present  circuit  technologies,  CMOS/ 

SOS  technology  provides  one  of  the  best  speed- 
power  products  for  system  integration  (Fig.  1). 
Its  low  speed-power  product,  typically,  a 
tenth  of  a  plcojoule,  is  attributed  mainly  to  the 
very  low  on-chlp  capacitances  in  t5q)ical  logic 
interconnections.  Also,  since  CMOS/SOS  is  a 
completely  static  logic  family  with  no  dc  paths 
to  ground,  dynamic  power  Is  a  direct  function 
of  switching  frequency.  When  applied  to 
microprocessors,  the  system  speed-power 
product  improves  even  more  because  only  a 
small  percentage  of  the  logic  elements 
(typically  10%  to  20%)  are  changing  state  dur¬ 
ing  a  given  instruction  execution.  Thus,  it  is 
possible  to  have  systems  with  10, 000  gates 
operating  at  instruction  execution  rates 
greater  than  10-MHz  which  dissipate  less 
than  1  W. 


Radiation  tolerance,  capable  of  meeting  the 
requirements  of  strat^c  weapons  systems 
and  long  duration  space  missions,  is  another 
key  feature  of  CMOS/SOS.  The  saphlre  Insula¬ 
ting  substrate  of  CMOS/SOS  provides  excellent 
Immunity  to  photocurrents  induced  during  high 
radiation  dose  rates.  The  intrinsic  transient 
dose  rate  hardness  of  CMOS/SOS  is  better  than 
10^®  rads  per  second.  Also,  the  total  dose 
hardness  of  CMOS/SOS  is  excellent.  Hard¬ 
ness  levels  better  than  10^  rads  are  being 
achieved  for  standard  circuits.  For  chips 
designed  with  specific  radiation-hardening 
circuit-design  techniques,  these  levels  are 
greater  than  10^  rads.  These  techniques^ 
Include  using  a  minimum  number  of  trans¬ 
mission  gates,  restricting  logic  circuits  to 
not  more  than  three  stacked  devices,  strap¬ 
ping  transistor  substrates,  and  selecting 
proper  device  ratios. 

Several  years  ago,  RCA  produced  a  high¬ 
speed  code  generator  using  the  hardening 
process  and  circuit-design  techniques  just 
described.  The  code  generator  performed  at 
a  rate  greater  than  10  MHz  after  exposure  to 
over  10®  rads  (Si).  The  availability  of  CMOS/ 
SOS  circuits  that  will  continue  to  perform  in 
hostile  radiation  environments  has  made  it 
possible  to  design  space  systems  that  operate 
for  long  periods.  One  example  of  a  system 
that  will  use  RCA's  radiation  hardened 
CMOS/SOS  technology  is  the  Air  Force 
sponsored  Fault  Tolerant  Spacebome  Computer. 

Other  characteristics  which  make  CMOS/SOS 
excellent  for  space  applications  Include: 

•  very  low  power:  no  static  power,  dynamic 
power  =  CV^f. 

•  High  device  density:  4  to  14  square  mils 
per  device  for  random  logic. 

•  High  noise  immunity:  45%  of  supply  voltage. 

•  Fully -static  logic  family. 

•  Mde  operating  voltage;  3  to  15  V. 

•  Wide  operating  temperature:  -55°C  to 
+125°C. 
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•  Veiy  Ugh  speed:  1  to  3  ns  gate  delays. 

DoD  estimates  that  the  total  life  cycle  costs 
of  electronic  equipment  over  a  twenty-year 
period  will  be  very  sensitive  to  power  dissi¬ 
pation  and  weight,  two  factors  for  which 
CMOS/SOS  in  an  optimum  technology.  TUs 
sensitivity  to  various  applications  is  indi¬ 
cated  below: 

Application  power  Weight 

Aircraft  $20/watt  $5000/pound 

Satellites  $2000/watt  $10,  OOO/pound 

Battery  Powered  $3000/watt 
Field  Equipment 

Clearly,  the  low  power  and  excellent  VLSI 
device  densities  of  CMOS/SOS  will  contribute 
to  significantly  lower  costs  for  future  aero¬ 
space  missions. 

Computer-Aided  Design 

One  of  the  most  important  attributes  of  CMOS/ 
SOS  is  its  compatibility  with  fully  automatic 
design  automation  layout  techniques.  Many 
aerospace  applications  place  such  rigid  con¬ 
straints  on  system  size,  weight  and  power 
that  VLSI  circuits  must  be  used.  Consequently, 
system  development  costs  are  increased  be¬ 
cause  they  are  spread  over  a  relatively  small 
number  of  components.  RCA's  low  cost  ap¬ 
proach  is  to  use  both  gate  universal  arrays 
(QUA)  and  standard  cell  approaches  for  cir¬ 
cuit  design.  Coiq)led  with  CMOS/SOS  technol¬ 
ogy,  they  provide  quick  turnaround  solutions 
for  meeting  the  VLSI  requirements  of  aero¬ 
space  applications.  RCA  is  currently  working 
on  extending  the  automatic  routing  programs 
associated  with  the  standard  cell  approach  to 
provide  an  automated  universal  array  (AUA) 
capability. 

One  notable  application  of  the  standard  cell 
approach  and  automatic  placement  and  routing 
design  automation  was  in  the  design  of  the  two 
ATMAC  microprocessor  chips  shown  in  Figs. 

2  and  3.  The  data  execution  unit  chip  contains 
4560  transistors  on  a  chip  measuring  255  x 
260  square  mils.  The  instruction  and  operand 


fetch  unit  chip  has  an  area  of  259  x  272  square 
mils  and  contains  approximately  4290  tran¬ 
sistors.  The  storage  registers,  consisting  of 
fairly  regular  logic  elements,  were  designed 
and  laid  out  using  normal  handcrafted  methods; 
the  random  logic  was  implemented  with 
standard  cells  for  automatic  placement  and 
routing.  The  ATMAC  program  would  not  have 
been  feasible  without  design-automation  tools, 
since  a  total  handcrafted  ATMAC  would  have 
been  at  least  five  times  more  costly  and  would 
have  delayed  the  program  an  additional  six  to 
twelve  months. 

In  conjunction  with  RCA  design-automation 
tools,  a  full  spectrum  of  simulation  and  check¬ 
ing  programs  is  used  in  the  design  of  CMOS/ 
SOS  VLSI.  Since  conventional  breadboarding 
techniques  are  inadequate  a  complete  package 
of  system,  logic  and  circuit  simulation  tools 
is  used  to  more  accurately  model  the  LSI 
circuits  under  consideration.  The  end  results 
are  components  that  work  correctly  the  first 
time  through,  with  reduced  design  time  and 
lower  non- recurring  costs. 

Advanced  Packaging 

The  extremely  low  power  dissipation  of  CMOS/ 
SOS  VLSI  makes  it  a  natural  choice  for  ad¬ 
vanced  hlgh-denslty  packaging  techniques 
which  can  optimize  the  size  and  weight  of 
electronic  systems.  As  previously  indicated 
the  cost  of  flying  electronic  equipment  is  very 
sensitive  to  weight.  Thus,  advanced  packaging 
approaches  can  be  quite  important  in  minimi¬ 
zing  total  life  cycle  costs  of  aerospace  systems. 

Conventional  packaging  of  VLSI  chips  mounted 
in  ceramic  dual-in-line  packages  on  pilnted- 
clrcuit  boards  produces  low  system  packaging 
densities.  The  problem  arises  because  VLSI 
chips  often  require  more  pins  per  chip  than 
MSI  and  SSI.  The  standard  64-pin  dual-in¬ 
line  package  used  in  many  of  our  VLSI  appli¬ 
cations  occupies  an  area  of  0. 9  inch  by  3. 4 
inches  to  mount  a  chip  that  is  less  than  |  inch 
on  each  side.  This  area  requirement  is  ex¬ 
tremely  inefficient  and  results  in  systems  that 
are  much  larger  and  heavier  than  necessary. 

An  advanced  packaging  approach  successfully 
used  by  RCA  is  a  thlck-fllm  ceramic 


Interconnect  board  approach  with  chips  mounted 
in  leadless  hermetic  chip  carriers.  ^  The  chip 
carriers  generally  have  pins  mounted  on  all 
package  edges,  utilizing  pin  spacings  of  30  to 
50  mils.  These  packages  are  reflow  soldered 
to  a  ceramic  multilayer  thick-film  intercon¬ 
nection  substrate.  Figure  4  shows  the  size 
reduction  obtained  when  the  10  ICs  comprising 
a  16-blt  ATMAC  microprocessor  and  its  asso¬ 
ciated  special  function  unit  were  repackaged. 
Figure  5  shows  a  close  up  of  the  subassembly. 
Use  of  the  substrate  and  chip  carrier  ap¬ 
proach  can  yield  iq>  to  a  10-to-l  size  reduction 
over  conventional  printed  circuit  board  and 
dual-ln-Une  package  techniques. 

Dense  memory  systems  are  an  important  seg¬ 
ment  of  any  processor-base  aerospace  elec¬ 
tronic  system.  Uhder  AF  contract,  RCA  has 
applied  CMOS/SOS  IK  RAM  chips,  leadless 
hermetic  chips  carriers,  and  the  thick-film 
ceramic  interconnect  approach  to  the  Navy's 
ISEM  packaging  format  to  produce  a  very 
dense  memory  system.  Figure  6  shows  one 
side  of  a  doublesided  substrate  which  contains 
64  RAM  chips  and  40  resistors  producing  a 
memory  module  containing  8K  words  x  16  bits 
in  a  volume  of  about  2,  25  cu  in.  Using  this 
format  and  RCA's  latest  4K  CMOS/SOS  RAMs 
the  module  would  have  a  total  storage  capability 
of  32K  words. 

ATMAC  MICROPROCESSOR 
ATMAC  Architecture 

Approximately  five  years  ago,  RCA  Advanced 
Technology  Laboratories  started  development 
of  a  very  high  speed  programmable  signal 
processor  for  real-time  applications  in  rigidly 
constrained  systems.  This  processor,  called 
ATMAC,  has  great  payoff  potential  in  high- 
throughput  oriented  applications  where  existing 
microprocessors  cannot  meet  the  system 
requirements. 

The  ATMAC  architecture  shown  in  Fig.  7  is 
capable  of  executing  a  full  computational 
repertoire  of  189  instructions,  with  throughput 
of  about  3  million  operations  per  second.  The 
total  amount  of  hardware  necessary  for  a  sys¬ 
tem  is  minimized  because  ATMAC  uses  bi¬ 
directional  buses  for  interconnections  with 


peripheral  equipments  and  subassemblies.  The 
bidirectional  bus  system  (Fig.  8)  reduces  the 
number  of  connections  and  allows  asynchronous 
communication  with  program  memory,  data 
memory,  I/O  devices,  and  special  function 
units  (such  as  a  hardware  multiplier  or  accu¬ 
mulator).  ATMAC  system  throughput  is  opti¬ 
mized  by  a  high  degree  of  functional  parallelism. 
Throi^hput  is  at  least  three  times  greater  than 
that  of  general-purpose  architectures  with 
equivalent  cycle  times. 

Mast  of  the  current  ATMAC  applications  are 
based  on  a  16-bit  processor  configuration  with 
a  hardware  16-blt  x  16-bit  multlpller/32-bit 
accumulator  special  function  unit.  The  com¬ 
plete  processor  assembly  shown  In  Fig.  5 
consists  of  10  CMOS/SOS  VLSI  chips  dissi¬ 
pating  less  than  1.2  watts  of  dynamic  power 
and  is  a  very  powerful  signal  processing 
module.  The  system  can  perform  basic  multl- 
plyAccumulate  oriented  operations  (the  basic 
operation  required  in  many  digital  signal 
processing  algorithms)  in  less  than  700  ns. 
per  pair  of  data  memory  operands.  In  addi¬ 
tion,  the  module  performs  very  efficient  fast- 
fourier  transforms  (FFTs),  which  allow  in¬ 
formation  to  be  extracted  from  signals  in  the 
time  domain  by  transforming  them  into  the 
frequency  domain.  The  modular  sections  of 
an  FFT  program  have  been  written  for  the 
ATMAC  microprocessor  so  that  the  execution 
time,  I/O  time,  scaling  time,  and  memoiy- 
size  requirements  for  both  the  program  and 
data  memory  can  be  estimated.  For  many 
applications,  the  execution  times  are  short 
enough  to  allow  real-time  FFT  processing 
(see  Table  I).  Results  show  that  ATMAC 
memory  requirements  for  a  moderate  number 
of  data  points  are  very  reasonable. 

The  power  dissipation  for  a  full  16-bit 
machine  configuration  is  less  than  3/4  W  at 
full  speed  and  10  V  operation,  while  its 
static  power  requirement  is  approximately 
10  mW. 

ATMAC  ^plications 

ATMAC  is  currently  being  used  in  several 
military  programs  having  high  performance 
and  low  power  requirements.  New  applications 
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are  being  reviewed  continually.  Ai^llcatlons 
considered  include  guidance  computers  for 
spacecraft,  missiles,  torpedoes,  and  RPVs; 
electronic  war&re  systems;  radar  applications, 
navigational  systems;  and  communication 
systems.  Descriptions  of  two  current  RCA 
programs  follow. 

Application  to  Speech  Processing 

A  narrowband  speech  prdcesslng  system  was 
ATMAC's  initial  target  application.  Here 
ATMAC  was  used  to  apply  linear  predictive 
coding  (LPC)  theory  to  speech  bandwidth  com¬ 
pression  for  digital  transmission  of  voice  over 
telephone  lines.  For  the  current  LPC  algori¬ 
thm  Implemented  on  ATMAC,  the  speech 
spectrum  Is  described  as  the  coefficients  of  a 
10th  order  all-pole  filter.  The  spectrum  of 
the  ln^)ulse  response  of  this  filter  closely 
approximates  the  envelope  of  the  original 
speech  spectrum.  Pole  values  are  calculated 
by  evaluating  an  autocorrelation  function  and 
performing  a  simple  recursion. 

The  ATMAC  implementation  of  the  LPC  sys¬ 
tem  is  shown  in  Fig.  9.  The  special  function 
unit  Is  a  16  X  16  bit  two's  complement  multi¬ 
plier  with  a  32-bit  accumulator.  The  ATMAC 
DMA  facility  Is  used  for  transferring  speech 
data  to  and  from  the  system.  Both  Input  and 
output  are  accomplished  with  a  single  DMA 
facility  by  Interleaving  input  and  output  data 
in  memory.  Execution  times  and  memory 
requirements  for  the  LPC  algorithms  (as¬ 
suming  a  70-ns  clock  cycle)  are  given  in 
Table  n. 

A  complete  speech  terminal  capable  of  both 
modem  and  linear  predictive  processing  can 
be  implemented  with  less  than  50  LSI  arrays. 
The  entire  system  could  be  housed  in  a  5  x  5  x 
10  Inch  package  with  power  dissipation  of  less 
than  5  W. 

Application  to  Sonar  Signal  Processing 

In  mid  1978,  RCA  assembled,  programmed, 
tested  and  demonstrated  an  ATMAC-based 
system  that  performs  adaptive  spatial  beam 
forming,  complex  translation,  and  digital 
filtering  for  processing  sonar  signals  In 
realtime.  ATL  recently  completed  fabrication 


of  the  full  scale  development  hardware  fpr 
deployment  (Fig.  i0).~  uses-a_diml_, 

ATMAC  processor  configuration  to  perform 
realtime  computations.  The  two  16-bit 
ATMAC  CPU's  interface  to  an  array  channel 
for  realtime  data  collection,  a  secure  com¬ 
munications  channel  for  transferring  processed 
data  to  a  shore  station,  and  a  console  channel 
for  system  initialization  and  debug.  The  CPU's 
perform  spatial  and  frequency  filtering,  and 
transmit  the  results  of  this  processing  to  a 
shore  station  via  a  UHF  satellite  link.  The 
realtime  processing  is  directed  via  commands 
received  from  the  shore  station  and  processed 
by  the  computer  system  while  realtime  proces¬ 
sing  is  proceeding. 

The  processing  functions,  execution  times  and 
memory  requirements  of  CPU-l  and  CPU-2 
are  listed  in  Tables  HI  and  IV.  This  dual¬ 
processor  ATMAC  system  in  Fig.  10  is  imple¬ 
mented  on  seventeen  6x7  inch  circuit  boards, 
occupying  less  than  0. 65  ft.  ^  and  dissipating 
less  than  10  W  total  power. 

RADIATION-HARDENED 
MICROPROCESSOR  CHIP  SET 

Description  of  the  Family 

In  1975,  the  Air  Force  Avionics  Laboratories 
embarked  upon  a  major  development  program 
with  RCA  to  develop  a  radiation-hardened 
microprocessor  chip  set  intended  for  emulating 
general-purpose  computers.  ®  This  is  a  flexi¬ 
ble  bit-slice  family  of  building  blocks  which 
provides  a  cost-effective  development  approach 
for  radiation  hardened  versions  of  key  micro¬ 
processor  families.  In  addition,  the  family 
provides  the  Air  Force  with  a  technology  to 
replace  obsolete  microprocessors  from  single¬ 
source  siqipllers  as  the  devices  become  un¬ 
supported. 

RCA  Is  e^qploring  numerous  architectures 
aimed  at  efficient  emulations  (high  performance 
with  reasonable  hardware  structures)  of  the 
various  tri-servlce  instruction  set  architecture 
standardization  programs.  In  all  of  these  pro¬ 
grams,  the  GPU  chip  set  offers  a  low-cost 
development  approach  for  high-technology 
improvements  in  existing  general-purpose 
computers  (microprocessors,  minicomputers 
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or  even  main-frame  processors),  hardware 
compatible  with  embedded  (small  size,  weight, 
and  power  dissipation)  processciFutlll^Soi^  ” 
and  substantially  improved  performance  and 
environmentally  tolerant  components.  A  key 
benefit  of  this  approach  is  that  substantial  im¬ 
provements,  in  basic  computing  devices  can 
be  realized  while  maintaining  compatibility  at 
the  user  instruction  set  level.  This  represents 
a  large  savings  of  existing  support  software 
tools  which  industry  and  DoD  estimate  to  be 
worth  over  $30  million  dollars  per  computer 
type. 

A  standard  military  computer  configured  in 
the  GP  U  chip  set  would  have  an  Immediate 
high  volume  market.  An  ideal  application  ex¬ 
ample  is  the  GPS  user  equipment.  The  em¬ 
bedded  processor  requirements  of  this  appli¬ 
cation  base  are  tailored  toward  a  militarized, 
small  size,  low-power,  radiation  tolerant, 
high  performance,  general-purpose  computer 
with  high  level  language  programmability.  A 
GPU- based  machine  standard  military  com¬ 
puter  would  be  an  ideal  solution. 

RCA  is  currently  under  contract  to  the  Air 
Force  Materials  Laboratory  (AFML)  to  de¬ 
velop  the  CMOS/SOS  manufacturing  technology 
for  the  Radiation  Hardened  Microprocessor 
chip  set.  Current  members  of  the  family 
Include: 

1)  General  Processor  IMt  (GPU).  An  8-blt 
slice  LSI  component  with  a  16  word 
register  file,  latches,  shifters,  an  arith¬ 
metic  logic  circuit  which  can  perform 
add's,  and's,  or's  and  compliment's. 

Parts  have  been  built,  and  can  perform 
register  to  register  operations  in  less  than 
250  ns. 

2)  Memory.  A  IK  RAM  Is  available  with  a 
100  ns  access  time.  4K  parts  may  also  be 
available  for  this  effort. 

3)  Microsequencers.  Two  microsequencers 
have  been  designed  and  are  now  being 
fabricated  by  RCA.  One  is  a  functional 
duplicate  of  the  2910.  The  other  is  a 
device  which  grew  from  the  GP  U  design  and 
the  desire  to  emulate  many  Instruction  sets 


with  one  GP  U  chip  set.  It  has  many  fea¬ 
tures  of  the  2910  and  the  2904. 

4)  8x8  Multiplier.  This  multiplier  has 
an  8  X  8  multiply  time  of  200  ns,  and 
is  concatenable.  Four  stages  of 
Booth's  algorithm  are  used  to  produce 
a  16-bit  result. 


5)  Gate  Uhlversal  Arrays  (GUA's).  Mask 
programmable  logic  arrays  have  been 
designed  to  provide  custom  interconnection 
between  the  LSI  integrated  circuits  of  the 
GPU  chip  set.  Four  sizes  are  available: 
182,  300,  452  and  632  gates  with  40,  48, 

64  and  64  l/O  pads  respectively. 

6)  ROM.  A  1024-blt  mask  programmable  is 
available  with  an  access  time  of  120  ns. 
This  ROM  is  designed  for  fast  control 
store  applications  and  has  CMOS  com¬ 
patible  outputs. 

Higher  density  RAMs  and  ROMs  compatible 
with  10-MHz  micro-cycle  operation  are 
available  within  RCA  and  will  eventually  be¬ 
come  part  of  the  AF  chip  set.  A  4k  static 
CMOS/SOS  RAM,  the  TCS  210,  was  used  in 
the  PDP-11  emulation  system  described  later 
and  is  being  used  in  several  RCA  military 
applications.  The  part  uses  5-mlcron  burled 
contact  design  rules  and  achieves  10-volt 
operation  with  an  access  time  of  less  than 
100  ns.  The  basic  layout  and  memory 
technology  are  being  appUed  to  development 
of  a  16K  static  CMOS/SOS  RAM  and  an  8K 
mask  program  ROM.  Both  have  access  times 
of  less  than  100  ns  and  are  compatible  with 
emulation  architectures.  Finally,  a  higher 
performance  GPU  chip  set  using  narrow  chan¬ 
nel  lithography  (4-micron  channel  lengths)  is 
oirrently  in  planning  states  and,  if  developed, 
the  components  could  provide  performance 
capabilities  in  excess  of  bipolar  at  a  fraction 
of  the  power  dissipation. 

Application  to  a  PDP-11  Emulation 

RCA  designed,  fabricated,  debugged,  and 
demonstrated  a  PDP-11  emulation 


demonstration  system  using  the  GPU  chip  set 
as  basic  building  blocks  (Fig.  11).  The  goals 
of  the  program  were  specifically  aimed  at 
showing  the  feasibility  of  emulating  commer¬ 
cial  or  standard  military  computers  with  the 
GPU  chip.  Also,  the  compatibility  of  existing 
support  software  (l.e. ,  debug  packages, 
assemblers,  editors,  etc. )  of  the  target  ma¬ 
chine  would  be  demonstrated  on  the  new  ma¬ 
chine  architecture.  The  ultimate  result  of  the 
program  would  show  the  technology  advantages 
of  CMOS/SOS  (substantially  improved  size, 
weight,  power,  and  performance)  In  a  military 
computer  application  base  while  being  com¬ 
patible  with  a  wealth  of  existing  siqiport  soft¬ 
ware.  The  complete  design,  fabrication  and 
debug  of  the  demonstration  system  was  funded 
on  a  veiy  modest  budget,  demonstrating  the 
low  development  cost  of  this  approach. 

The  PDP-11/40  standard  instruction  set  was 
chosen  as  the  target  machine  for  the  emulator 
demonstration  because  of  its  widespread  use 
within  the  military.  However,  the  specific 
selected  target  machine  was  not  the  primary 
focus  of  the  demonstration  since  the  design 
techniques,  and  development  cycles,  are 
common  to  any  selected  target  machine  within 
the  minicomputer  class.  Demonstration  of  the 
feasibility  and  resulting  improvements  of  this 
emulation  approach  for  military  computers 
was  the  key  goal. 

Figure  12  shows  a  general  block  diagram  of 
the  PDP-11  emulation  architecture.  The  sys¬ 
tem  is  oriented  around  a  two-bus  organization 
including  a  16-blt  unldlrection  address  bus 
and  16-blt  bidirectional  data  bus.  The  sys¬ 
tem  uses  PDP-11  compatible  memory  mapped 
I/O,  but  bus  control  and  electrical  Interfaces 
were  designed  specifically  for  this  architecture 
and  are  not  UNIBUS  compatible. 

Three  key  CMOS/fiOS  building  blocks  were 
used  in  the  demonstration.  Including:  the  GPU, 
CMOS/SOS  2910  microprogram  sequencer, 
and  RCA  TCS  210  4k  static  RAM.  In  the 
demonstration  system,  the  logic  requirements 
for  these  auxiliary  functions  were  satisfied 
with  off-the-shelf  TTL  and  CMOS  MSI  and  SSI 
components,  but  they  could  later  be  replaced 
with  GUAs  or  standard  cell  custom  LSI  in 
actual  productizing  of  architecture.  The 


architecture  Incorporated  parallelisms  and 
pipelining  wherever  reasonably  possible  to ' 
achieve  a  high  throughput  capability. 

To  project  the  capabilities  and  potential  of  an 
all-LSI  version  of  the  emulation  architecture 
paper  designs  at  the  detailed  reglster/bus  level 
have  been  performed.  Two  new  chip  tsrpes 
would  be  needed.  Each  would  be  relatively 
simple  to  develop  and  amenable  to  either  GUA 
or  standard  cell  custom  LSI  Implementation. 
The  first  type  would  be  a  four-bit  slice  through 
the  data  path  siq>port  functions  with  four  chips 
required  for  the  16-bit  system,  while  the 
other  chip  type  would  int^rate  all  the  control 
siq>port  functions  augmenting  the  2910.  With 
these  two  new  types  and  a  512  word  x  8  bit 
CMOS  ROM,  an  all-LSI  system  would  require 
a  total  of  only  18  LSI  components,  as  shown 
below: 


2910  microprogram  sequencer  -  1 

GPU  chips  -  2 

GPU  support  chips  (new  type  #1)  -  4 

Sequencer  siq>port  chip  (new  type  #2)  -  1 

Microprogram  and  mapping  ROM  chips  -10 
TOTAL  18 


In  addition  to  small  size,  the  all-LSI  emulator 
would  have  a  djmamic  power  dissipation  at  full 
operating  frequency  of  approximately  2-3  watts 
and  a  static  dissipation  of  less  than  50  milli¬ 
watts. 

Detailed  performance  comparisons  of  the 
CMOS/SOS  emulator  approach  and  the  various 
DEC  PDP-11  models  were  made.  Table  1 
shows  a  weighted  average  performance  com¬ 
parison  of  the  RCA  CMOS/SOS  Emulator  and 
the  projected  all-LSI  emulator  against  the 
vadous  PDP-11  models.  The  instruction  mix 
used  in  the  comparisons  was  taken  from  C.G. 
Bell  et  al,  "Computer  Engineering".  As 
shown  in  the  table,  the  breadboard  system  was 
slightly  better  than  the  PDP-ll/40  while  the 
all- LSI  emulator  would  be  very  close  to  the 
dec's  top  of  the  line,  the  PDP-ll/45,  with  a 
throughput  potential  of  approximately  790  KOPs. 

CONCLUSIONS 

CMOS/SOS  technology  has  proven  its  capability 
and  viability  for  real-time  data  and  signal 
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processing  applications.  The  material  pre¬ 
sented  in  this  paper  has  shown  how 
the  CMOS/SOS  technology  can  be  applied  to 
real-time  signal  processing  applications,  all 
with  low  power  reqidrements.  The  flexlMllty 
of  the  technology  allows  system  designers  to 
configure  processor  architectures  as  simple 
as  arithmetic  processors,  and  as  complex  as 
distributed  processor  networks.  Further,  the 
technology  provides  designers  with  the  capa¬ 
bility  to  produce  equipment  that  will  withstand 
the  high  radiation  environment  encountered  in 
satellites  and  missiles.  In  summary, 

1.  The  CMOS/SOS  technology,  fully  auto¬ 
mated  custom  LSI  design  capability, 
coupled  with  advanced  packaging  techniques 
provides  an  ideal  technology  base  for  future 
space  missions. 

2.  The  RCA  ATMAC  microprocessor  Is  an 
example  of  an  ideal  signal  processor 
primitive  for  computationally  intensive 
aerospace  applications. 

3.  The  AF  sponsored  GPU  chip  set  provides 
an  excellent  building  block  family  for 
emulating  commercial  and  standard  DoD 
data  processing  instruction  set  architectures. 
The  resulting  implementations  provide 
substantially  improved  performance,  size, 
weight  and  power  dissipation  over  existing 
Implementations  as  well  as  Intrinsic  en¬ 
vironmental  tolerances  that  are  being  de¬ 
manded  In  future  space  missions. 

RECOMMENDATIONS 

1.  DoD/NASA  should  develop  through  Industry 
a  detailed  CMOS/SOS  technology  data  base 
required  by  space  processor  designers. 

Elements  of  the  data  base  should  include: 

•  CMOS/SOS  reUablllty 

•  CMOS/SOS  radiation  hardening 

•  CMOS/SOS  basic  building  blocks  circuits 
and  chips  such  as  RAMs  and  ROMs 

•  Fault-tolerant  concepts  that  are  imple- 
mentable  in  CMOS/SOS  technology 


•  Space  packaging  concepts 

•  Software  development  tools  that  can  be  used 
to  simulate  space  processors  requirements 

•  Space  experiments  that  allow  gathering 
Information  that  supports  space  processor 
designs 

2.  DoD/NASA  space  applications  require 
technology  performance  levels  that  exceed 
commercial  requirements.  Therefore, 
DoD/N  ASA  must  fully  develop  sources  for 
the  highly  reliable  radiation  hardened 
technologies  like  CMOS/SOS  that  will  be 
required  to  satisfy  the  space  processor 
requirements  of  the  1980's  and  90's. 

3.  DoD/NASA  should  Initiate  the  development 
of  a  CMOS/SOS  microprocessor  version 
of  MIL-STD-1750.  The  speed,  density 
and  CAD  tools  of  CMOS/SOS  have  matured 
to  the  point  that  a  VLSI  version  of  MIL- 
STD-1750  can  be  implemented  with  low 
risk  and  within  a  reasonable  development 
cycle.  Such  a  device  would  substantially 
Improve  today's  avionics  processors  and 
would  have  broad  applicability  in  embedded 
avionics  systems.  In  addition,  a  viable 
product  of  this  nature  would  enhance  the 
feasibility  of  the  Air  Force's  computer 
standardization  programs  and  provide 
tremendous  life-cycle  cost  benefits  for 
future  avionics  missions. 
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TABLE  I.  FFT  EXECOTION  TIMES  FOR  AN 
ATMAC  MICROPROCESSOR o 
SFU  MULTIPUER 


No.  of 

Data  Memoxy 

Complex  Points 

Words 

TABLE  11.  PROGRAM  MODULES  <i  ALLOTMENTS 
FOR  SPEECH  PROCESSING  SYSTEM 


TABLE  m.  PROGRAM  MODULES  &  ALLOTMENTS 
FOR  ATMAC  CPU  1 


Maximum 

Execution 

Time 

(PS) 


Algorithm 


Array  Interface 
Control  and 
Data  Shading 
Beamformlng 
Cosine  and  Sine 
Value  Computation 
Low  Power  Idle 
Receiver  Interface 
Interprocessor 
Communication 


•8-blt  words 


TABLE  IV.  PROGRAM  MODULES  fc  ALLOTMENTS 
FOR  ATMAC  CPU  2 


Maximum 

Execution 

Time 

(ps) 


1  Memory  Requirements 

Program 

(24-blt 

words) 

Data 

(16-blt 

words) 

DMA  Channel  Servicing 
Beam-Band  Pair 
Connection 
Complex  Translation 
Scheduling  -  Filters, 
Sat  Comm,  Other  I/O 
Low-Pass  Filtering 
Output  Formatting  ti 
Data  Buffering 
Transmitter  Interface 
Low  Power  Idle 
Interprocessor 
Communication 
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Fig.  2.  Data  Execution  Unit. 


Comparison  of  Packaging  Techniques 
The  area  occupied  by  the  10  64-pin 
ICs  is  reduced  5  to  1  by  using  lead¬ 
less  hermetic  chip  carriers. 


Fig.  5.  ATMAC  Microprocessor  and  SFU 

Multiplier  Assembly.  The  four  large 
packages  contain  the  16-bit  micro¬ 
processor.  This  assembly  measures 
2.5  X  5.  0  inches. 


Fig.  8.  ATMAC  System  Block  Diagram.  The 
bidirectional  bus  allows  as5mchronous 
communication  between  the  micro¬ 
processor  and  memory,  l/o  or  spec 
special  function  units. 


Fig.  6,  70-Chip  Double-sided  Hybrid  in 

Improved  Standard  Electronic  Module 
(ISEM)  Format. 


Fig.  7.  ATMAC  Microprocessor  Block 

Diagram.  A  two-chip  configuration 
is  used  to  allow  parallel  operations. 


Fig.  9.  Laboratory  Model  Speech- Bandwidth 
Compression  System.  High-quality 
speech  resulting  from  linear  pre¬ 
dictive  coding  techniques  can  be 
demonstrated  with  this  system. 


Fig.  10.  Sonar  Signal  Processor  Full-Scale 
Development  Equipment. 


Fig.  11.  PDP-11  Emulation  System.  This 
demonstration  unit  uses  the  CMOS/ 
CMOS/SOS  GPU  and  Microprocessor 
developed  by  the  Air  Force  as  well 
as  RCA's  4K  CMOS/SOS  static  RAM. 


Fig.  12.  Block  Diagram  of  GPU-based 
PDP-11  Emulator. 


DoD  PROBLEHS  AND  OEVELOPHENT  THRUSTS  IN  5ISNAL  PROCESSORS 

•  DoO's  FUTURE  WEAPON  SYSTEMS  REQUIRE  DRAMATIC  IMPROVEMENTS  IN  PROCESSING  THROUGHPUT. 

•  HOST  FUTURE  SYSTEMS  ARE  SIGNAL-PROCESSING-ORIENTED. 

«  SIGNAL  PROCESSORS  HAVE  BEEN  TRADITIONALLY  IMPLEMENTED  USING  CUSTOM  DEDICATED  APPROACHES. 

-  HORRENDOUS  DEVELOPMENT  COSTS  AND  SCHEDULES. 

•  INFLEXIBLE  AND  SPECIALIZED  TO  SPECIFIC  APPLICATIONS. 

-  DIFFICULT  TO  INSERT  NEW  TECHNOLOGY. 

•  COMMERCIAL  SEMICONDUCTOR  INDUSTRY  HAS  NOT  ADEQUATELY  ADDRESSED  THIS  TECHNOLOGY 

•  OoO  AND  DARPA  ARE  FUNDING  SEVERAL  PROGRAMMABLE  SIGNAL  PROCESSING  PROGRAMS. 

•  IM  ESTABLISHED  $210  MILLION  TRI-SERVICE  VHSIC  PROGRAM  TO  SPECIFICALLY  IMPROVE  DoD 
SIGNAL  PROCESSING  CAPABILITIES. 


OoO  FUTURE  SYSTEMS  W.LL  REQUIRE  ADVANCED  PROCESSING  PRIMITIVES 


•  >  90t  OF  OoO  FUTURE  SYSTEMS  WILL  Bt  CONFIGURED  WITH  EHBEDOEO  PROCESSORS. 

•  OoO  SYSTEMS  ARE  DEMANDING  10-100X  THROUGHPUT  IMPROVEMENTS. 


SYSTEM 

PLATFORM 
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1.000 
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Tightly-Coupled  ATMAC  System  Architecture 


•V 


349 


CMOS/SOS  is  the  Ideal  Technology  for 
Standard  Military  Computers 

e  COMPLETE  CPUs  WITH  ECL  SPEEDS  ARE  REALIZABLE  ON  A  FEW  NUMBER  OF 
CHIPS 

e  VERY  LOW  POWER  CV^f 

e  VLSI  DENSITIES:  2.000  GATES  TODAY.  20.000  GATES  WITHIN  5  YEARS 
e  VERY  HIGH  SPEED;  3  5ns  TODAY;  Ins  WITHIN  5  YEARS 
e  intrinsic  and  ENHANCED  RADIATION  HARDNESS 
TRANSIENT  RATE  -  5  X  10^0  raDS/SEC. 

TOTAL  DOSE  -  10®  RADS 
e  ALL  THE  NORMAL  ADVANTAGES  OF  CMOS 
e  COMPATIBLE  WITH  DA  LAYOUT  TECHNIQUES 


CAD/DA  System  Required  for  Successful 
System  and  Support  VLSI  Development 

RCA  IS  AN  INDUSTRY  LEADER  IN  DA  FOR  VLSI 
e  PIONEERED  STANDARD  CELLS  &  MP2D 

«  ACCURACY  &  QUICK  TURNAROUND  tS  ESSENTIAL  FOR  OoO  VLSI 

•  SEVERAL  CMOS;’SOS  STANDARD  CELL  FAMILIES  EXIST  -  7  MIL  (HIGH  SPEED) 

4.2  MIL  (HIGH  DENSITY).  6.3  MIL  (RAO  HARD).  NARROW  CHANNEL  (IN  DEVELOPMENT) 

RCA  ACTIVELY  USES  HIERARCHIAL  LAYOUT  APPROACHES 
e  CUSTOM 
e  STANDARD  CELL 

•  AUTOMATIC  universal  ARRAY 

RCA'S  INTEGRATED  CAO/DA  SYSTEM  IS  ESSENTIAL 


Simulation  and  CAD  Essential  for 
VLSI  ADP  Implementation 

CONVENTIONAL  BREADBOARDING  NOT  APPLICABLE 

•  TOO  EXPENSIVE 

•  NO  FEASIBLE  1  TO  1  IMPLEMENTATION 

•  LONG  DEVELOPMENT  CYCLE 

SIMULATION  AT  ALL  STATES  OF  ADP  DEVELOPMENT 

•  SYSTEM  SIMULATION  (BUS.  REGISTER  LEVEL!  -  ALSIM.  APL.  N.mPc.  ISPL 

•  LOGIC  LEVEL  SIMULATION  -  LOGSIM.  MIMIC.  ETC 

•  CIRCUIT  LEVEL  -  RCAP,  SPICE 

DESIGN  AUTOMATION  AND  OTHER  CAD 

•  CIRCUIT  CHECKING  -  CRITIC 

•  CHIP  CHECKING  -  TACC 

•  MULTI-PORT  TWO  DIMENSIONAL  PLACEMENT  AND  ROUTING 

•  MACRO  LOGIC  AUTOMATIC  LAYOUT 

•  FAULT  SIMULATION  AND  TEST  SEQUENCE  GENERATORS 


RCA  Has  A  Variety  of  CMOS /SOS  Memories 
to  Support  Standard  Military  Computers 


MEMORY 

ORGANIZATION 

SPEED 

DYNAMIC  POWER 

STATUS 

RAMs 

TCS-210 

1Kx4 

100  nt. 

20  mw/MHi 

AVAILABLE 

TCS^ISS 

4Kx1 

50  ns. 

IS  mtw/MH2 

AVAILABLE 

TCS-130 

4Kx4 

100  ns. 

56  mw/MHz 

IN  EVALUA 
TION 

ROMs 

TCS-190 

8Kx8 

100  ns. 

20  mw/MH^ 

IN  DESIGN 

TA10682  ; 

1Kx8 

250  ns. 

12  mw/MHs 

IN  DESIGN 

CADDAS  Data  Management 


16K  CMOS/SOS  RAM 
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CMOS/SOS  Reliability 


Microprocessor  Features 


1024  by  1-Blt  RAM 


Lot 

Lifa-Ttst 

Voliaet 

SwnpM 

Sut 

Functional 

Failures 

1000  Hrt 

ToUl 

Hours 

Tested 

Functional 

Failuras 

1 

10 

12 

0 

3624 

0 

2 

10 

19 

0 

3288 

0 

3 

10 

18 

0 

3624 

0 

4 

10 

14 

0 

5802 

1(5466  hr) 

5 

10 

4S 

0 

1000 

0 

6 

s 

21 

0 

3316 

0 

7 

5 

16 

0 

3316 

0 

8 

5 

» 

0 

3316 

0 

9 

5 

13 

0 

5730 

0 

10 

5 

77 

0 

1000 

0 

11 

10 

77 

0 

1000 

0 

12 

5 

77 

0 

1000 

0 

■  TOTAL  OT  '.ampler  •  W 

■  TOTAL  MOuBi  Of  T(5T1NC  •  H^.UO 

■  SCBHMO  DlVlCIi  If  i;iiOC 

■  fAlLOlC  BATE  •  «0; 

e  T  •  U50C.  0,541/1000 

e  T  .  %iO(;  0  0004. ’lOCO  tAC’UAt'O',  Of  1  I  e.i 


TIROS-N  use  Data  for  CMOS/SOS 

CDP  1821  RAM  I 


DEVICES 

HOURS 

DEVICES  HOURS* 

FAILURES 

306 

11,064 

3.3a  X  10® 

0 

•AS  OF  18  JAN.  1980 


■  T  -  10*C 
•  Voo •  5V 

■  MTBF- 3.65x10^  HRS..  60%  CONFIDENCE 

■  DEVICES  HOURS  DATA  TO  18  JAN.  1980 


•  189  INSTRUCTIONS 

•  ALL  INSTRUCTIONS  EXECUTED  IN  ONE  MACHINE  CYCLE 

•  APPROXIMATELY  3  MILLION  INSTRUCTION  PER  SECOND 

•  DATA  Cr  PROGRAMS  IN  SEPARATE  MEMORIES 

•  DATA  PATH  EXPANDABLE  IN  8  BIT  MODULES 

•  DATA  b  PROGRAM  MEMORY  ADDRESSING  INDEPENDENTLY 

EXPANDABLE  IN  8  BIT  MODULES 

•  ARCHITECTURAL  FEATURES  PERMITTING  MULTIPLE  PARALLEL 

OPERATIONS 

•  DMA  CHANNEL 

•  SYNCHRONOUS  Et  ASYNCHRONOUS  1-0  OPERATIONS 

•  PROGRAM  INTERRUPT 

•  SYSTEM  CONNECTIONS  PROVIDED  FOR  SPECIAL 

FUNCTION  UNIT  (SFUl 

•  CMOS/SOS  IMPLEMENTATION 

•  LOW  POWER  CONSUMPTION  - 16  BIT  CPU  3/4W 


Advanced  Autonomous  Array 
|A^|  Concept 


CMOS/SOS  Emulation  of  PDP-11 


•STANDARD  INSTRUCTION  SET  OF  PDP  11/40 


•  PARALLELISM  AND  AUXILIARY  HARDWARE  PROVIDED  FOR  HIGH  THROUGHPUT 

emulation 


•  COMPATIBLE  WITH  AN  ALL  CMOS/SOS  LSI  APPROACH 


A  WORD  LENGTH  EXPANDABLE  TWO 
CHIP  TYPE,  8-BIT  WIDE  SLICE, 
MICROPROCESSOR.  HIGH  PROCESSING 
SPEED  ACHIEVED  THROUGH 
OVERLAPPED  OPERATIONS.  THE  USE  OF 
SPECIAL  FUNCTION  UNITS  AND  1-0 
TRANSFERS  USING  A  DIRECT  MEMORY 
ACCESS  CHANNEL 

CMOS/SOS  CONSTRUCTION  USED. 


•  Firmware  or  hardware  floating  point  can  easily  be  added 

•  Based  on  gpu  &  cmos/sos  z9io 

RCA's  PDP-11  Emulation  System  is  Fully  Operational 
with  Three  Key  CMOS/SOS  VLSI  CHIPS 

•  GPU  RALU  J 

I  PART  OF  AF 

•  CMOS/SOS  2910  MICROPROGRAM  K  raD  HARD  MICRO 

SEQUENCER  j  PROCESSOR  FAMILY 

•  TCS-151  4K  RAM  I 


THE  IMPACT  OF  LARGE  SCALE  INTEGRATION 


ON  SPACECRAFT  ARCHITECTURE 
PhUlip  E.  Holt 
PROJECT  ENGINEER, 

THE  AEROSPACE  CORPORATION 


In  order  to  discuss  the  subject  in  this  very  brief 
overview,  some  early  definitions  are  helpful: 

Architecture:  A  collection  of  data  processing 
elements  which  are  configured  to  perform  a 
specific  task  or  set  of  tasks. 

Autonomy:  The  ability  of  a  spacecraft  to  operate 
in  a  sustamed  manner  with  no  external  command 
or  control  influences;  perhaps  with  inputs  from 
its  own  sensors. 

Throughput:  Simply  a  measure  of  the  rate  at 

which  jobs  are  completed  by  a  processing  system. 

Spacecraft  architecture  has  been  impacted  by 
LSI  technology  in  two  major  ways:  Indirectly, 
largely  as  a  result  of  the  explosive  growth  in 
ground-based,  commercial  computing  cababil- 
ities;  and  directly,  due  mostly  to  economies  in 
cost,  power,  weight,  size  and  in  reliability 
considerations. 

The  indirect  impact  results  from  today's  ability 
to  plan  and  execute  complex,  long-term  space 
missions.  Such  planning  has  demanded  the 
design  of  high -reliability  processing  capabilities. 
In  order  to  achieve  these  high  reliabilities, 
direct  implementation  of  traditional  designs  has 
been  accomplished  using  LSI  components. 

What  has  LSI  really  bought  us  in  terms  of  space¬ 
craft  computer  architecture?  The  answer  seems 
to  be,  in  large  part,  that  reliability  considera¬ 
tions  have  been  the  major  driver.  These  consi¬ 
derations  have  resulted  in  architectures  which 
treated  the  following  items: 

o  Modular  Designs 
o  Redundancy 
o  Fault  Isolation 
o  Majority  Voting 
o  Multiple  Fault  Tolerance 

However,  in  general,  with  the  above  exceptions, 
we  have  generally  succeeded  only  in  implemen¬ 
ting  architectures  of  the  1960's  with  1970'8  LSI 
components. 

The  results  of  several  recent  surveys  of  today's 
space -qualified  computers  produces  a  composite 
picture  of  the  "average"  space  computer: 

o  Speed  300  KIPS 

o  Memory  64  K  Words 

o  Power  50  Watts 

o  Weight  30  Pounds 

o  Size  600  Cubic  inches 


Obviously,  any  individual  space  computer  will 
not  have  identically  the  above  characteristics, 
but  one  can  be  assured  it  will  not  differ  from 
these  characteristics  by  an  order  of  magnitude. 

Furthermore,  if  one  wants  to  buy  a  space  com¬ 
puter,  one  quickly  discovers  that  the  selection 
is  limited.  For  a  specific  application  one 
would  be  fortunate  to  find  more  than  a  handful 
of  candidates.  In  total,  there  are  probably 
fewer  than  30  models  of  space-qualified  com¬ 
puters  available  today. 

Space  computers  fall  into  either  of  two  broad 
categories: 

o  General  Purpose  -  these  are  software 

programmable  and  used  typically  iu  applica¬ 
tions  such  as  Attitude  Control  and  so-called 
Housekeeping. 

o  Special  Purpose  -  these  are  often  hard  wire 
programmed  to  fimction  essentially  as 
programmed  sequencers. 

Substantially  all  space  computer  d'^signs  are 
five  to  ten  years  old:  the  equivalent  of  two  or 
three  computer  generations. 

What  areas  have  been  impacted  by  LSI? 
Certainly,  the  most  obvious  relief  has  come  in 
power,  weight  and  size.  Cost  has  reduced 
substantially,  tempered  of  course  by  signifiCcint 
expense  associated  with  parts  qualification. 
Memory  technology  has  been  the  truly  outstand¬ 
ing  performer  since  it  intrinsically  lends  itself 
to  LSI  techniques.  Perhaps  most  important  of 
all  is  the  development  of  the  microprocessor 
and  its  successor,  the  computer  on  a  chip. 

These  devices  have  yet  to  impact  space  in  any 
significant  manner. 

What  areas  will  be  impacted  soon?  The  guess 
here  is  that  the  continued  proliferation  of  a 
wide  variety  of  LSI  components,  coupled  with  an 
extension  of  commercial  advances  (e.  g.  ,  bus- 
structured  architectures),  and  new  applications 
will  result  in  a  variety  of  new  spacecraft 
architectures.  Today's  fundamental  reliance 
on  Von  Neumann  machine  orgcuiization  (Ca.  1946) 
will  prove  inadequate  for  sophisticated  and 
complex  problems.  Mass  storage  LSI  techniques 
will  open  new  avenues  for  the  system  architect. 
On  the  other  side  of  the  coin,  "testability"  will 
present  significant  challenges.  Testability  and 
design-to-test  techniques  will  probably  evolve 
into  separate  and  distinct  disciplines. 

What  areas  need  to  be  driven?  Two  significant 
problems  need  a  solution:  radiation  hardening 
and  a  wide  operating  temperature  regime. 

Neither  of  these  problems  will  be  attacked 
because  of  commercial  pressures.  Space  users 
will  have  to  provide  the  incentive  for  solutions. 
Two  other  areas  need  to  be  driven:  the  search 
for  and  definition  of  the  proper  applications,  and 
significant  improvement  in  software  productivity. 
No  easy  solution  is  on  the  horizon  for  either  of 
these  items. 

As  a  matter  of  fact,  it  is  possible  that  in-space 
computing  must  always  be  well  behind  the  leading 
edge  of  technology  because  of  the  following 
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factors: 

o  Space  processing  requirements  are  unique 

o  Hardware  components  are  derated  because 
of  the  operating  environment 

o  There  are  few  and  demanding  customers 

o  Obsolescence  rate  -  time  to  field  a  given 
technology,  coupled  with  long  mission  life, 
approaches  that  of  a  computer  generation  life 
span. 

What  are  the  future  requirements  ?  Space  compu¬ 
ting  and  LSI  technologies  need  further  advances 
in 

o  Still  higher  reliability 

o  Radiation  hardened/Qualified  parts 
inventory 

o  Reprogrammability 
o  Higher  speeds  by  order  of  magnitude 
o  Non-volatile  mass /bulk  storage 

There  are  several  fundamental  questions  that 
need  early  answers,  la  there  a  general-purpose 
architecture  which  could  adapt  to  most  mission- 
specific  requirements  ?  Probably  not,  without 
some  quantum  jump  in  throughput  capability. 

Moat  new  applications  that  are  now  being  consi¬ 
dered  require  sheer  speed  of  operation  to  sustain 
throughput.  However,  it  is  possible  that  a 
distributed  architecture  may  be  useful  for  a 
particular  class  of  problems. 

Will  commercial  developments  alleviate  space 
parts  problems ,  particularly  with  regard  to 
radiation,  temperature  regime,  and  vibration? 

It  does  not  appear  likely  that  these  problem  areas 
will  be  addressed  because  of  commercial  market 
pressure.  There  is  no  driving  commercial  need. 

Will  space  developers  continue  to  go  it  adone? 

It  is  now,  and  has  been  the  case  for  some  time, 
that  each  space  user  develops  and  procures 
hardware  without  considering  cooperative 
partnership  arrangements.  Possibly,  the  require¬ 
ments  are  so  unique  that  going  it  alone  is  the 
curse  of  the  business. 

The  bottom  line  so  far  seems  to  indicate  that, 
in  general,  adequate  technology  will  be  available 
for  near-term  requirements,  and  the  real  ques¬ 
tion  is  whether  or  not  there  will  be  a  determined 
effort  on  the  part  of  space  users  to  apply  this 
technology.  This  is  not  to  say  that  there  won't 
be  some  shortfall  in  technology  unless  dollars 
are  thrown  at  the  problems.  As  mentioned 
previously,  these  areas  are  most  likely  to  be 
radiation,  temperature  and  software  productivity. 

What  does  the  future  hold?  Briefly,  the  following 
factors  seem  self-evident: 

o  LSI  technology  will  allow  advanced  architec¬ 
tures 

o  Processors  will  move  closer  to  the  sensors 

o  Architectures  will  probably  be  distributed  in 
nature  until  tiiroughput  increases  by  several 
orders  of  magni  ..de 


o  Processing  applications  will  grow  in  number 
and  complexity 

o  Microprocessors  will  proliferate  throughout 
the  spacecraft 

o  On-board  processing  will  demonstrate  high 
cost  effectiveness  by  reducing  the  required 
ground  support,  by  simplifying  ground  node 
configurations,  and  by  reversing  the  trend 
of  increasing  ground  mission  operations  cost 
and  complexity 

The  On-Board  System  architecture  issues  are 
complex.  We  need  to  know  how  much  processing 
can  be  done  on  board.  This  appears  to  be  a 
strict  function  of  budget  (cost,  schedule,  power, 
weight  and  size)  iutd  dependent  upon  the  applica¬ 
tion.  We  must  trade  oii  on-board  versus  ground 
operations  by  establishing  firm  system  require¬ 
ments  which  then  allow  partitioning  of  the 
problem.  Partitioning,  combined  with  technology, 
allow  solving  "bite^sise"  problems  so  that 
timing  and  sizing  estimates  are  realizable. 

After  all  the  above  things  are  accomplished,  we 
need  to  solve  the  problem  of  getting  "there". 
"There"  being  the  place  where  we  are  accomp¬ 
lishing  in  space  those  processes  which  are  coat 
effective.  This  is  probably  the  toughest  problem 
to  solve.  Any  approach  must  resvilt  in  a  flexible 
solution  which  provides  adequate  design  margin, 
room  for  expansion  and  reasonable  contingency. 
There  is  no  single  best  solution:  getting  there 
must  necessarily  involve  reasonable  evolution¬ 
ary  steps.  All  of  this  can  be  accomplished 
only  by  a  dedicated,  long-term  development 
plan. 

What  can  we  conclude  ? 

1.  The  "average"  spacecraft  computer  is  slow, 
small,  application  limited  amd  old. 

2.  The  direct  impact  of  LSI  on  architecture 
has  been,  but  need  not  be,  minimal. 

3.  Operational  considerations  will  be  the  driver 
which  impacts  architecture  because  of  the 
requirement  to  operate  in  a  stressed  environ¬ 
ment  and  the  need  to  simplify  ground  stations. 

Here  are  some  recommendations,  none  of  which 
are  easily  accomplished: 

1.  Identify  what  tomorrow's  in-orbit  processors 
should  look  like. 

2.  Identify  the  technology  which  must  be  driven 
to  get  there. 

3.  Be  bold  and  bite  the  bullet;  find  the  money; 
establish  a  plan;  and  set  imaginative,  but 
realisLc  goals. 


NEW  DEVELOPMENT  IN  MICRO¬ 
ELECTRONIC  TECHNOLOGY 


3.  LSI  Design  Tools 


SUMMARY  OF 

ISSUES  AND  RECOMMENDATIONS 

1.  LSI  Product  Adequacy 
Issue 

Some  existing 'Class  "S"  micro- 
circuit  requirements  are  inade¬ 
quate  and/or  inappropriate  for 
LSI. 

Recommendations 

o  Develop  a  set  of  product  assur¬ 
ance  requirements  and  test 
methods  which  are  effective  and 
appropriate  for  LSI,  e.  g.  , 

Visual  inspection 
Design  for  testability 
Acceptance  testing 

2.  Use  of  Commercial  Parts 
Issue 

It  is  often  cost  effective  to  use 
commercial  mainstream  parts  but 
such  parts  may  not  meet  the  low 
power,  radiation  tolerance  require¬ 
ments  for  space  programs.  It  is 
important  to  identify  a  complete 
family  of  Class  "S"  parts  for  use 
by  contractors. 

Recommendations 

©Identify,  qualify  and  make  avail¬ 
able  a  complete  family  of  low 
power  Class  "S"  parts  which  are 
radiation  hardened. 

o  Develop  specific  rad-hard  parts 
where  necessary  to  complete  the 
family,  e.  g. ,  static  RAM,  non¬ 
volatile  memory,  microprocess¬ 
ors,  etc. 

o  Develop  a  set  of  constrained 
design  tools  that  will  permit 
supplementing  this  family. 


Issues 

Limited  availability  of  constrained 
design  tools  for  LSI  parts. 

Recommendations 

o  Fund  a  national  design  center  to 
provide  approved  design  tools 
and  design  verification  parts  for 
all  agencies  and  contractors. 

o  Establish  qualified  radiation- 
hardened  sets  of  low  power  LSI 
building  blocks  including  master 
slice  gate  arrays  and  standard 
cells. 

o  Identify  a  baseline  process 
facility  to  build  prototype  parts. 

4.  Radiation  Requirements 
Issue 

Radiation  requirements  limit  the 
use  of  LSI  technology  in  space. 

Recommendations 

o  Establish  realistic  and  consistent 
radiation  requirements. 

o  Develop  lot- sample  testing 
techniques  for  hardness  assur¬ 
ance. 

o  Fund  programs  to  develop  parts 
and  processes  capable  of  meet¬ 
ing  the  necessary  levels  of  rad¬ 
iation  tolerance  as  well  as  the 
performance  and  low  power 
requirements  of  space  missions. 

5.  Radiation  Data  Availability 
Issue 

JPL  has  gathered  a  great  deal  of 
data  on  device  susceptibility  to 
radiation,  however,  that  data  is 
not  available  in  published  form  for 
use  by  industry  and  government. 


Recommendation 

o  NASA  should  see  to  it  that  this 
valuable  data  base  is  made 
available  in  a  timely  fashion. 
When  published,  it  should  be  put 
into  GIDEP  for  easy  access. 

Availability  of  Memories 

Issue 

Limited  availability  of  low  power 
radiation  tolerant  static  random 
access  memories. 

Recommendations 

o  Develop  low  oower  static  RAMs 
capable  of  10^  RAD(Si)/sec  upset 
and  10^  RAD(Si)  total  dose. 

IK  Manufacturing  Technology 
4K  Development 
16K  R  &  D 

o Specify  and  test  for  suscepti¬ 
bility  to  cosmic- ray  induced 
single  event  upsets. 

Availability  of  Microprocessors 

Issue 

Limited  availability  of  radiation- 
tolerant  microprocessors. 

Recommendations 

o Qualify  to  Class  "S"  and  radia¬ 
tion  tolerance  requirements 
those  microprocessors  now  on 
the  SD/NASA  standard  parts  list 
and  those  plaqned  for  that  list. 

o  Procure  small  quantities  of  each 
type  for  qualification  test  by 
vendor  and  make  these  available 
to  agencies  and  contractors  after 
qualification  is  complete. 


8.  Packaging 

Issue 

High  density  packaging  approaches 

are  needed  for  spacecraft. 

Recommendations 

o  Finalize  Class  "S"  hybrid  speci- 
cations  to  insure  hybrid  reli¬ 
ability. 

o  Encourage  the  use  of  hermetic 
chip  carrier  parts  build  to  Class 
"S"  standards  and  mounted  on 
multilayer  ceramic  substrates. 


WORKSHOP  F 

CRAFTSMANSHIP  WORKSHOP 
AN  APPROACH  TO  PERFECTION 

SESSION  F-3  MANUFACTURING  READINESS  AND  CONTROLS 


Chairman 

Stan  Chamberlain 
Manager,  Quality  Assurance 
General  Dynamics-Convair 

Transition  from  Design  to  Manufacturing 
Manufacturing  Readiness  Industry 
Manufacturing  Readiness  Government 

Control  of  Critical  Items 


Paul  Dalton 
Martin  Marietta 

Dick  Sipe 
Boeing 

Ed  Houston 
Ballistic  l^issile  Office 
Norton  AF  Base 

John  Wickham 
General  Dynamics-Convair 


Manufacturing  and  Inspection  Software 


Bill  Chumbley 

Bell  Helicopter-Textron 
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CRAFTSMANSHIP  WORKSHOP 
AN  APPROACH  TO  PERFECTION 

COORDINATOR 
MIKE  BUTLER  MDAC 

SESSION  F-1  CONTRACT  REQUIREMENTS  AND  APPLICATION 
Walt  Carrion 

Chief  of  Engineering  Service  Division 
NASA,  Goddard  Space  Flight  Center 


Introduction 

Walter  Carrion 

Realistic  Contract  Requirements 

Robert  E.  Balmat 

Rockwell  International 

Counter  Productive  Requirements/ 

Specifications 

Dr.  Altman 

United  Technology 

Directions  and  Needs  in  Craft  Training 

James  P.  Mitchell 

Bureau  of  Apprenticeship  & 
Training 

Barriers  of  Productivity 

Max  Lehrer,  RCA 

Contractor  Selection/Past  Performance 

Colonel  B.  Weiss 
Aeronautical  Systems  Div. 
Wright  Patterson  AF  Base 

Contractor  Assessment  During 

Program  Office  &  CAO  Audits 

Maj.  D.  Austin 

NASA  Headquarters 

Contractor/Contract  Administration  Office  Audits 

J.  Rytlewski 

DSASR/LA 

SESSION  F  2  WORKMANSHIP  AND  PRODUCTIVITY 


Chairman 

Colonel  Bill  Bagwell 

Director  of  Contracting  and  Support 

AF  Space  &  Millile  Test  Organization 


The  Personal  Initiative  Aspect  of  Craftmanship  Norm  Wilson,  Rockwell 


Quality  vs.  Schedule  Tradeoff 


Art  Haueschle 
Honeywell 


Training  for  Craftsmanship 


Dick  Egelston 
Aerospace 


Workmanship  Standards 


Jack  Reberry 
AF,  Space  Division 


Craftsmanship  vs.  Collective  Bargaining 


D.  M.  Verrastro 
Martin  Marietta  Aerospace 


CRAFTSMANSHIP  WORKSHOP 
INTRODUCTION 

SESSION  I  -  CONTRACT  REQUIREMENT 
AND  APPLICATION 


Walter  J.  Carrion 

Chief,  Engineering  Services  Division 
Goddard  Space  Flight  Center 


Good  Morning,  and  welcome  to  the  first 
session  of  the  Craftsmanship  Workshop. 

I  hope  everyone  has  come  to  work-not 
just  listen,  but  to  participate. 

Our  aim  in  the  next  few  days  is  to 
examine  how  we  do  business  together;  how 
we  can  Improve  productivity  and  how 
Industry  and  government  can  work  better 
as  a  team. 

Inflation  and  world  competition  Is  com¬ 
pelling  us  to  re-examine  ourselves  and 
make  Improvements  in  order  to  reverse  the 
nation's  downward  trend  In  its  rate  of 
productivity  growth.  The  rate  of  pro¬ 
ductivity  growth  in  the  United  States  in 
the  period  1963  to  present  has  been  the 
lowest  of  11  major  industrial  nations. 

Productivity  Gains 
Average  Annual  Increase  In  Output 
Per  Man-Hours  In  Manufacturing 

Japan . 

Netherlands . 

Sweden . 

Belgium . 

Italy . . . 

France . 

West  Germany . 

Switzerland . 

Canada . 

United  Kingdom . 

United  States . 

Source:  U.S.  Department  of  Labor 

The  United  States'  present  yearly  pro^th 
rate  now  approaches  zero.  For  many  years 
America's  technological  superiority  has 
enabled  it  to  dominate  the  industrial 
market,  especially  In  aerospace,  aviation, 
electronic  and  machine  tools.  This  too. 

Is  being  seriously  challenged  by  our 
foreign  competitors  who  are  receiving 
financial  support  from  their  governments 


to  help  turn  our  technological  advantage 
to  zero.  Therefore,  it  is  of  the  utmost 
importance  to  this  nation  that  government 
and  industry  continuously  re-evaluate  it¬ 
self  In  order  to  make  the  most  efficient 
use  of  our  available  resources,  both 
dollars  and  manpower. 

Better  utilization  of  dollars  and  manpower 
means  greater  productivity,  more  business 
for  industry  and  greater  technological 
advantage  for  this  country  to  compete  in 
the  world  market. 

There  are  many  factors  affecting  produc¬ 
tivity.  They  range  from  unrealistic 
contract  requirements  and  specifications 
to  impractical  design  and  manufacturing 
techniques,  to  Improper  workmanship  stan¬ 
dards,  to  lack  of  personnel  initiations 
and  skilled  craf tmanshlp . 

In  the  next  3  days,  we  invite  all  of  you 
to  openly  discuss  with  us  any  problems 
you  experience  that  are  counter  productive 
to  developing  a  final  product.  Today's 
agenda  will  be: 

FIGURE  2 

The  success  of  this  workshop  depends  on 
your  participation. 

Back  home  in  my  own  Division,  I  hold  a 
Round  Table  Discussion  each  month  to  pulse 
the  Division  to  find  out  what  is  bothering 
the  troops,  where  in  the  Division  we  are 
falling  down,  and  what  new  ideas  are  "out 
there."  Each  Branch  sends  two  employees 
(different  employees  each  month)  to  the 
Round  Table  Discussion  to  discuss  what  is 
on  their  minds. 

Ladles  and  gentlemen,  I  assure  you,  it's 
a  eye-opener  and  a  very  effective  tool  in 
bringing  to  light  problems  I  didn't  even 
know  existed.  It  brought  to  my  attention 
things  that  are  counter  productive,  manu¬ 
facturing  problems,  morale  problems, 
safety  problems,  mlsunderstandlne  of 
policies,  misunderstanding  of  what  is 
needed,  etc. 

I  look  at  this  entire  workshop  as  a  Round 
Table  Discussion.  However,  the  only  Round 
Table  Discussions  that  are  of  any  value 
are  those  in  which  the  employees  come  in 
and  openly  and  freely  participate.  Just 
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as  I  tell  my  employees — I  can't  solve  a 
problem  if  I  don't  know  about  It  or 
understand  it.  Government /industry 
problems  cannot  be  resolved  unless  we 
are  all  aware  of  the  problems.  So,  I 
encourage  all  of  you  to  participate  in 
bringing  to  light  items  that  are  counter 
productive,  and  suggest  realistic  recom¬ 
mendations  to  improve  our  use  of  the 
nation's  manpower  and  dollar  resources. 

Our  mission  reminds  me  of  the  proverbial 
donkeys — each  having  their  own  desires. 

FIGURE  3 

I  am  positive  that  we  are  as  smart  as 
the  donkeys 

FIGURE  4 

Let's  share  our  resources. 

FIGURE  5 

Let's  pull  together. 

and  when  we  summarize  the  results  of 
today's  workshop,  we  will  move  on  to 
greater  productivity  by  working  closer 
together  as  a  team,  as  we  better  under¬ 
stand  one  another's  problems. 


CRAFTSKAMSUZP  UORiLSKOP 


-  An  Approach  To  Parfactloa  • 


TUESDAY:  Seaaion  I  •>  Contract  Requlrenanta  and  Appllcatiooa 


TOPICS: 
0800  -  0815 
0815  -  0845 


0845  -  0915 

0915  -  0945 

0945  -  1000 
1000  -  1130 

1130  -  1300 
1300  -  1325 

1325  -  1350 

1350  -  1445 

1445  -  1500 


A.  Introduction  by  Chalraan — Walter  J.  Carrion.  NASA/Coddard  Space  Flight  Center 


B.  Realistic  Contract  Requirements 

a  Specific  Workmanship  Requiremanta 
e  Workmanship  Requirement  (How  Imposed) 

-  System/CI  Specification 

-  SOW  Compliance 
a  Tailoring 

C.  Counter  Productive  Requiremants/Speeif Icaclona 
e  ZdentlflcaClon 

0.  Directions  and  Heeds  in  Craft  Training 


BREAK 

Panel  Discussion 

1000  -  1010  Barriers  To  Productivity 


LUNCH 

E.  Industry/Covernment  Role  in  Mission  Assurance 
a  Warranties 

F.  Contractor  Selection 
a  Past  Performance 

G.  Contractor  Asseasmenc  During  Program 

1350  -  1415  Program  Office  4  CAO  Audita 
1415  •  1445  Contractor/C^ntract  Administration 
Office  Audits 

BREAK 


Robert  E.  Balmat,  Controller 

Space  Operations  &  Satellite  Sys.  Division 

Rockwell  International 


David  Aleman r  Senior  Vice  President 
United  Technologies 

James  P.  Mitchell*  Administrator 
Bureau  of  Apprenticeship  4  Training 


Messrs.  Balmat*  Altman,  Mitchell,  Lehrer 

Max  Lehrer,  Division  Vice  President 
Govt.  Systems  Division,  RCA 


Chester  Hardy,  Project  Engineer 
General  Dynamics 

Col.  Bernard  Weiss,  Deputy  for  Contracts  4 
Kfg.  Aeronautical  Sys.  Divison 
Wright'Patterson  Air  Force  Base 


MaJ .  David  Austin,  NASA  Headquarters 
John  Rytlewskl,  Quality  Assurance  Staff 
Specialist,  DCASR/LA 


LSQO  -  1830  H.  Round  Tsble  Discussion 
1630  -  1640  I.  Closing  Remarks 


Walter  J.  Carrion,  Chief,  Lnglneering 
Services  Division,  Goddard  Space  Flight  Center 
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REALISTIC  CONTRACT  REQUIREMENTS 
Robert  E.  Balmat 

The  presentation  will  discuss  contract 
implication  for  mission  assurance.  The 
discussion  will  explore  issues  dealing 
with  the  contracts  statement  of  work, 
schedule,  human  resources,  facilities  and 
equipment,  financial  resources,  manage¬ 
ment  emphasis  and  communication  effect¬ 
ing  mission  assurance.  Mr.  Balmat's 
comments  are  drawn  from  current  program 
experience  on  NASA  and  Air  Force  Systems 
Division  (AFSD)  spacecraft  programs  and 
observations  of  trends  in  today's  business 
climate  viewed  from  the  perspective  of 
a  prime  contractor  whose  mission  assurance 
motives  ultimately  lead  to  a  profitable 
and  health  business. 


ACQUISITION  PRACTICES 


IN  SUPPORT 


OF  MISSION  ASSURANCE 


ACQUISITION  FACTORS 

•  PROGRAM  STRUCTURE 

•  SELECTION  PROCESS 

•  CONTRACTUAL  ENVIRONMENT 

•  PROGRAM  EXECUTION 

ACQUISITION  OBJECTIVES 

•  REALISTIC  PROGRAM 

•  COMPETENT  CONTRACTOR 

•  SOUKO  PROGRAM  EXECUTION 


ELEMENTS  OF  MISSION  ASSURANCE 


COST  AND 
SCHEDULE 
PERFORMANCE 


REALISTIC  PROGRAM 

•  REAL  "CUSTOMER’VREAL  NEED 

•  ADEQUATELY  DEFINED  SOLUTION 

•  APPROPRIATE  PERFORMANCE 
SPECIFICATIONS 

•  COST  EFFECTIVE  AND  AFFORDABLE 


•  REALISTIC  SCHEDULE 


COMPETENT  CONTRACTOR 

•  DESIGN  CONCEPT 

•  TECHNICAL  CAPABILITY 

•  MANAGEMENT 

•  COMPETITIVE  PRICE 

•  MOTIVATION 

SOUND  PROGRAM  EXECUTION 

•  CONTRACTOR  PERFORMANCE 

•  GOVERNMENT  PROGRAM  MANAGEMENT 

•  PRODUCTIVITY 

CONTRACTOR’S  OBJECTIVE 
EQUITABLE  RETURN  ON  INVESTMENT 

•  RISKS 

•  INVESTMENT 

•  ALTERNATIVES 

PROFESSIONAL  REPUTATION 

MAINTAIN  LONG  TERM  BUSINESS  PROSPECTS 


OBJECTIVES 

QOVERNMENT  INDUSTRY 

•  REALISTIC  PROGRAM 

•  COMPETENT  CONTRACTOR  FAIR  RETURN 

•  SOUND  PROGRAM  EXECUTION 

•  GOVERNMENT  ^ 

•  CONTRACTOR  ^ 

SOME  PROBLEMS  IN  PRACTICE 

REALISTIC  PROGRAM 

•  STATEMENT  OF  WORK 

•  SCHEDULE 

COMPETENT  CONTRACTOR 

•  CONTRACT  STRUCTURE 

•  PROFITABILITY 

SOUND  PROGRAM  EXECUTION 

•  PROGRAM  MANAGEMENT 

•  PRODUCTIVITY 

SOME  PROBLEMS  IN  PRACTICE 

ISSUE 

•  INADEQUATE  STATEMENT  OF  WORK 

OBSERVATIONS 

•  INSUFFICIENTLY  SPECIFIC 


•  CONTRADICTORY 


SOME  PROBLEMS  IN  PRACTICE 

CONSEQUENCES 

•  INACCURATE  BIDS 

•  INACCURATE  BUDGETS 

•  SCOPE  NEGOTIATIONS 

•  PROGRAM  DELAYS 

•  JEOPARDIZED  MISSION 

SOLUTION 

•  MANAGEMENT  EMPHASIS 

•  DRAFT  RFP’S 


SOME  PROBLEMS  IN  PRACTICE 

ISSUE 

•  UNREALISTIC  SCHEDULES 
OBSERVATIONS 

•  UNFORESEEABLE  REQUIREMENTS 

•  RFP  DELAYS 

•  LACK  OF  PLANNING 

•  LENGTHENING  LEAD  TIMES 


SOME  PROBLEMS  IN  PRACTICE 

CONSEQUENCES 

•  MISSION  RISKS 

•  HIGH  RISK  CONTRACTS 

•  HIGHER  COSTS 

SOLUTION 

•  PLANNING 

•  LONG  LEAD  PROCUREMENT 

•  AUTHORIZED  INVENTORY  COMMITMENTS 


SOME  PROBLEMS  IN  PRACTICE 

ISSUE 

•  INAPPROPRIATE  CONTRACT  STRUCTURE 

OBSERVATIONS 

•  FIXED  PRICE  VS.  COST  TYPE 

•  GEN.  SLAY  INITIATIVES 

•  CONTRACTOR  CONSIDERATIONS 

•  RISK  VS.  POTENTIAL  RETURN 

•  INVESTMENT  REQUIRED  AND  INTEREST 
EXPENSE 

•  ALTERNATIVES  AND  CAPACITY 
LIMITATIONS 


SOME  PROBLEMS  IN  PRACTICE 

CONSEQUENCES 

•  FEWER  BIDDERS 

•  GREATER  MISSION  ASSURANCE  RISKS 

•  HIGHER  BIDS 

SOLUTION 

•  USE  COST-TYPE  CONTRACTS  WHERE 
APPROPRIATE 


SOME  PROBLEMS  IN  PRACTICE 

ISSUE 

•  CONTRACTOR  PROFITABILITY 
OBSERVATIONS 

•  GOVERNMENT  BUSINESS  MUST  COMPETE 

•  RISK  AND  INVESTMENT 

•  RECENT  DEVELOPMENTS  INCREASE  RISK 

•  INTEREST  RATES 

•  FIXED  PRICE  CONTRACTS 


•  LIMITED  UPSIDE  POTENTIAL 


SOME  PROBLEMS  IN  PRACTICE 

CONSEQUENCES 

•  LOWER  RELATIVE  ATTRACTIVENESS  OF 
GOVERNMENT  BUSINESS 

•  REDUCED  EMPHASIS  BY  DIVERSIFIED  FIRMS 

•  REDUCED  INVESTMENT 

SOLUTION 

•  APPROPRIATE  CONTRACT  STRUCTURE 

•  AWARD  AND  INCENTIVE  ARRANGEMENTS 

•  LIQUIDATION  PROVISIONS 

SOME  PROBLEMS  IN  PRACTICE 

ISSUE 

•  PROGRAM  MANAGEMENT 
OBSERVATIONS 

•  INCONSISTENT  AND  UNTIMELY 
PROGRAM  DIRECTION 

•  CONTRADICTORY  PROGRAM  DIRECTION 

•  INFORMAL  OR  DELINQUENT 
PROGRAM  ADMINISTRATION 

•  LOSS  OF  MISSION/SOW  INTEGRITY 

SOME  PROBLEMS  IN  PRACTICE 

CONSEQUENCES 

•  DELAYS 

•  HIGHER  COST 

•  POSSIBLE  LOSS  OF  PROGRAM 

•  CONTRACTOR/CUSTOMER  CONFLICT 
SOLUTION 

•  STRONG,  UNIFIED  CUSTOMER  PROGRAM 
MANAGEMENT 

•  MISSION  -  ORIENTED  PROCUREMENT 


SOME  PROBLEMS  IN  PRACTICE 

ISSUE 

•  FORCES  AFFECTING  PRODUCTIVITY 
OBSERVATIONS 

•  LEGISLATED  SOCIAL  REQUIREMENTS 

•DIRECT  OPERATIONAL  INEFFICIENCIES 
•DIRECT  COST  IMPACT 
•  ADMINISTRATIVE  EXPENSE 

•  FUNDING  BELOW  EFFICIENT  LEVELS 

•  WAGE  CONTROLS 

•  CONFLICTING  OR  OVERLAPPING  GOVERNMENT 
REQUIREMENTS 


SOME  PROBLEMS  IN  PRACTICE 

CONSEQUENCES 

•  HIGHER  COST 

•  PROGRAM  DELAYS 

SOLUTION 

•  MULTI  YEAR  FUNDING 

•  O 


SUMMARY 

MISSION  ASSURANCE  IS  AFFECTED  BY 

•  PROGRAM  STRUCTURE 

•  SELECTION  PROCESS 

•  CONTRACTUAL  ENVIRONMENT 

•  PROGRAM  EXECUTION 

GOVERNMENT  AND  INDUSTRY  OBJECTIVES  ARE  SIMILAR 
GOVERNMENT  INDUSTRY 

•  REALISTIC  PROGRAM 

•  COMPETENT  CONTRACTOR  FAIR  RETURN 

•  PROGRAM  EXECUTION 


OPPORTUNITIES  FOR  IMPROVEMENT 

PROCUREMENT  PROCESS 

•  PLANNING 

•  STATEMENT  OF  WORK 
•SCHEDULE 

CONTRACTOR  DISINCENTIVES 
•RISK 

•INADEQUATE  RETURNS 
PROGRAM  EXECUTION 

•  STRONG.  CONSISTENT 
GOVERNMENT  MANAGEMENT 

PRODUCTIVITY  INHIBITORS 


CONCLUSION 

ACQUISITION  PRACTICES  CAN  ENHANCE  MISSION 
ASSURANCE 

•  REALISTIC  AND  WELL  DEFINED 
REQUIREMENTS 

•  REASONABLY  ATTRACTIVE  BUSINESS 
OPPORTUNITY 


•  STRONG  MISSION  ORIENTED  PROGRAM 
EXECUTION 


SOME  FACTORS  ARE  BEYOND  OUR  CONTROL 
OPPORTUNITIES  ARE  REAL 
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COUNTER-PRODUCTIVE  PRACTICES 
IN  PROCUREMENT 

David  Altman 

Senior  Vice  President-Technical 
Chemical  Systems  Division, 
United  Technologies 


It  is  a  privilege  to  address  this  con¬ 
ference  on  improving  mission  success 
because  of  its  great  importance  to  the 
aerospace  business  in  a  period  of 
declining  purchasing  power  and  tight 
budgets.  I  also  welcome  the  opportunity 
to  express  some  personal  convictions  on 
the  subject  which  have  evolved  over 
three-plus  decades  in  this  sometimes 
unpredictable  but  always  exciting 
business . 

To  start  out  with  fundamentals,  a  suc¬ 
cessful  procurement  can  be  defined  as  one 
in  which  the  product  performs  as  intended 
within  the  specifications,  and  at  a 
reasonable  cost.  When  viewed  from  this 
simple  definition,  I  can't  help  but  be 
impressed  with  the  overriding  signifi¬ 
cance  of  the  technical  aspects  of  our 
aerospace  programs  and  of  the  numerous 
cases  of  highly  publicized  overruns  and 
schedule  delays  resulting  from  perform¬ 
ance  fai lures --beyond  the  level  of 
reasonable  expectation.  What  I  would 
like  to  consider  on  this  occasion  is  the 
degree  to  which  both  practices  and 
attitudes  in  the  procurement  process  can 
influence  the  effectiveness  in  obtaining 
a  reliable  product  that  assures  mission 
success . 

I  would  like  to  address  some  of  the  more 
fundainental  deficiencies  that  frequently 
occur  during  the  procurement  process, 
both  in  contractor  selection  and  in 
program  monitoring.  Most  of  these 
deficiencies  are  really  matters  of 
attitude  and  state  of  mind  rather  than  of 
action,  but  are  nevertheless  the  result 
of  counter-productive  practices. 

Obviously,  if  the  procurement  process  has 
selected  the  "right"  contractor  with  the 
right  design,  the  chances  are  much  better 
that  the  contract  will  be  successfully 
performed.  The  challenge,  therefore,  is 
how  to  select  the  "right"  contractor  and, 
more  important,  since  that  is  not  always 
possible  in  practice,  what  are  the  other 


actions  that  should  be  taken  so  that  an 
acceptable  product,  which  is  affordable, 
will  be  delivered  with  an  "average" 
contractor.  Since  one-half  of  our 
industry  is  composed  of  "average"  con¬ 
tractors,  this  becomes  an  important 
challenge. 

Second  Source  or  Backup  Contracts 
In  reviewing  some  of  the  major  programs 
that  have  suffered  from  severe  overruns 
due  to  technical  difficulties,  I  am 
impressed  with  the  fact  that,  although 
procurement  may  have  made  the  best 
contractor  selection  based  on  the  data 
available  at  the  time,  an  effective  means 
of  assuring  mission  success  would  have 
been  to  purchase  insurance  in  the  form  of 
a  backup  contract,  i.e.,  second  source. 
This  approach  is  particularly  important 
where  there  are  critical  subsystems 
pushing  the  state-of-the-art.  Dual  con¬ 
tracts  for  such  critical  subsystems  should 
be  carried  through  critical  design  review 
or  to  that  point  where  further  technical 
risk  appears  minimal.  Several  early  pro¬ 
curements  have  employed  this  approach, 
one  example  being  the  Minuteman  missile. 

An  important  prerequisite  is  the  ability 
to  identify  a  high  risk  subsystem  which 
has  the  capability  of  being  a  show- 
stopper.  Even  when  such  critical  areas 
are  identified,  the  pressures  in  the 
current  political  climate  are  enormous  to 
stay  with  only  one  contractor  because  of 
the  apparent  cost  savings  in  the  near 
term.  A  dollar  saved  now  is  more  impor¬ 
tant  than  three  or  four  dollars  spent  in 
two  years.  But  is  it?  We  talk  about 
life  cycle  cost,  but  are  we  willing  to 
abide  by  the  conclusions  of  such  an 
approach?  Despite  our  conviction  of  its 
fundamental  correctness,  we  usually 
rationalize  the  apparent  low  cost, 
because  it  is  easier  to  sell. 

Fear  of  Failure 

The  desire  to  have  development  programs 
that  are  devoid  of  "failures"  has  become 
more  and  more  acute  in  recent  years  for 
many  reasons,  not  the  least  of  which  is 
the  increasing  tendency  to  conduct  devel¬ 
opment  efforts  in  a  "fishbowl"  where  an 
experimental  failure  in  a  development 
test  is  associated  with  a  deficiency  in 
the  program.  This  situation  is  sympto¬ 
matic  of  a  fundamental  misconception  of 
how  a  development  program  in  a  high 
technology  area  should  be  run.  Where 
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there  are  no  "failures,"  even  incipient 
ones,  we  rob  ourselves  of  that  critical 
knowledge  of  the  boundaries  of  safe  and 
reliable  operation.  Indeed,  a  develop¬ 
ment  program  without  failures  bears 
considerable  scrutiny,  and  should  be 
viewed  with  more  than  a  little  suspicion. 
I  recall  some  15  years  ago  when  visiting 
the  engineering  manager  at  Pratt  &  Whit¬ 
ney,  we  discussed  his  philosophy  in 
engine  development.  When  I  asked  him 
how  things  were  going,  with  a  worried 
expression  he  told  me  he  had  considerable 
concern  over  an  engine  in  development 
because  in  the  first  40  engine  tests, 
there  were  no  failures,  giving  him  a 
great  feeling  of  apprehension.  As  a 
result,  he  was  planning  to  expand  the 
operational  conditions  to  create  at  least 
an  incipient  failure  to  bolster  his 
confidence.  This  engine,  incidentally, 
has  since  had  a  very  successful  opera¬ 
tional  experience. 

Perhaps  the  most  distressing  aspect  of 
the  fear  of  failure  during  development  is 
the  psychological  impact  on  both  the 
program  managers  and  their  superiors 
which  inhibits  the  effective  conduct  of 
the  program.  All  development  efforts, 
expecially  those  involved  in  advancing 
the  state-of-the-art,  will  experience 
test  failures*  If  the  contingency  of 
having  a  failure  is  not  planned,  a 
frenetic  atmosphere  not  conducive  to  a 
clear,  rational  approach  needed  for  the 
solution  will  result  if  a  failure  occurs. 
Many  failures  are  due  to  contractor 
deficiency,  but  more  frequently,  they 
result  from  lack  of  proper  planning 
provoked  by  real  or  imagined  competitive 
pressures  during  negotiations  when  cost 
considerations  are  paramount.  The  fear 
of  failure  has  also  been  exaggerated  by 
the  short  duty  tour  of  government  pro¬ 
gram  managers  whose  personal  careers 
could  be  damaged  by  a  program  which  has 
the  "appearance"  of  failures. 

Another  distressing  feature  of  the  fear 
of  failure  is  the  inhibition  it  creates 
in  pursuing  innovative  and  aggressive 
approaches  in  new  technological  areas, 
historically  the  cornerstone  of  our 
national  growth.  We  need  to  rethink  our 
approach  to  development  programs  and 
encourage  an  attitude  that  permits  us  to 
advance  the  technological  barriers  by 
openly  recognizing  that  development 


"failures"  must  accompany  excursions  into 
areas  bordering  on  the  unknown. 

Bid  Cost  Versus  Technical  Excellence 
All  of  you  have  undoubtedly  had  discus¬ 
sions  on  this  well-known  subject  of  the 
relative  significance  of  technical  excel¬ 
lence  versus  promised  cost  during  the 
award  of  a  program.  On  this  issue,  I 
would  like  to  restrict  my  comments  to 
development- type  contracts,  especially 
those  of  high  technological  content.  A 
common  misconception  with  some  procure¬ 
ment  people  is  that,  if  the  group 
proposals  are  "technically  acceptable," 
the  only  other  discriminator  is  the  cost. 
Since  a  technically  unacceptable  proposal 
usually  requires  an  obvious  deficiency, 
the  technically  acceptable  field  can  have 
a  wide  spread  of  technical  capability  and 
with  no  further  discriminant  placed  on 
the  so-called  technically  acceptable 
proposals,  a  disproportionate  emphasis  is 
placed  on  cost.  The  procurement  is 
frequently  reduced,  in  a  cost-reimburse¬ 
ment-type  contract,  to  selecting  the 
lowest  "promised"  cost.  All  of  the 
forces  that  bear  on  the  so-called  cost 
competition  tend  to  artificially  force 
the  cost  down.  The  budgeting  process 
does,  competition  does,  the  Government 
negotiator  does,  even  the  regulations  do, 
as  there  are  no  sanctions  against  under¬ 
bidding,  only  against  overbidding.  There 
are  fines  and  penalties  for  intentionally 
overbidding  (by  overstating  cost  elements) 
but  only  when  the  underbidding  is  extreme 
is  a  potential  contractor  thrown  out  for 
incredible  costing. 

As  the  not-so-surprising  result  of  this 
situation,  major  development  programs 
frequently  suffer  serious  overruns.  The 
underbid  program  is  then  performed  in  an 
environment  that  fosters  unreasonable 
short  cuts  and  insufficient  contingency 
for  anticipated  problems.  The  result  is 
usually  a  program  costing  more  than  one 
which  is  properly  costed  initially. 

What  can  be  done  to  reduce  the  hazard  of 
program  failures  due  to  unrealistic  low 
bids?  Eliminate  or  at  least  modify  those 
practices  that  generate  undue  emphasis  on 
bid  cost  rather  than  on  quality.  In 
particular,  a  much  more  restricted  use 
should  be  made  of  best  and  final  offer 
procedures  which  tend  to  reduce  the 
competition  to  an  auction. 
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As  a  final  comnent  on  this  important 
subject,  the  customer  must  be  wary  of 
not  contributing  to  an  "unreasonable" 
program  by  his  actions  during  a  compet¬ 
itive  negotiation.  During  this  phase, 
the  relentless  pressure  on  the  contractor 
when  caught  up  in  the  competitive 
atmosphere  is  to  reduce  price  and 
promise  more  work  at  a  higher  level  of 
performance.  The  result  is  usually  an 
overrun  and  delayed  program  due  to 
technical  difficulties.' 

Contributing  Issues 
In  addition  to  these  three  primary 
issues,  I  would  also  like  to  comment  on 
several  secondary  practices  that  can 
lead  to  inefficiencies  in  the  conduct  of 
a  program.  These  have  to  do  with  both 
excessive  management  and  direction  of  a 
program,  a  chronic  contractor  complaint. 

I  will  say  at  the  outset  that  such 
practices  have  arisen  for  legitimate 
reasons  in  programs  where  the  contractor 
was  in  difficulty.  However,  as  is 
characteristic  in  a  bureaucratic  system, 
these  practices  become  insti tutional ized 
with  the  result  that  they  are  frequently 
exercised  where  the  intended  benefit  is 
more  than  offset  by  the  inefficiency  that 
results  from  technical  management  atten¬ 
tion  being  diverted  from  areas  of  greater 
importance. 

One  of  the  counter-productive  practices 
I've  noted  is  a  tendency  toward  prema¬ 
ture  comnitment  to  the  "illities."  The 
early  phases  of  a  major  development 
program  represent  a  sensitive  period 
when  maximum  technical  and  management 
attention  should  be  focused  on  establish¬ 
ing  the  optimum  technical  baseline 
design.  It  is  well  known  that  major 
system  programs  are  frequently  preceded 
by  an  advanced  development  phase,  the 
purpose  of  which  is  to  demonstrate  and 
validate  a  technical  approach  and 
organizational  competence  to  carry  out 
the  program.  Such  contracts  are  usually 
unfettered  by  the  "illities"  associated 
with  full-scale  development,  such  as 
maintainability,  reproducibility,  human 
engineering,  and  configuration  management. 
This  approach  is  a  proper  one  since  the 
maximum  attention  can  now  be  placed  on 
technical  concept  and  hardware  produci- 
bility  with  changes  made  as  required 
within  a  reasonable  cost.  Carrying 
along  the  same  argument,  when  full-scale 


development  is  initiated,  full  implemen¬ 
tation  of  the  "illities"  should  be 
avoided  until  some  reasonable  point, 
such  as  the  critical  design  review. 

This  practice  would  permit  concentrating 
the  available  resources  on  the  most 
important  aspects  of  the  program  in  the 
early  phases  when  the  designs  are  set, 
to  optimize  both  the  probability  of 
mission  success  and  cost  effectiveness. 

Another  area  in  procurement  that  could 
bear  modification  is  overspeci fi cation 
of  the  product.  This  comes  about  in  two 
ways;  one  by  the  use  of  irrelevant 
specifications  which  is  usually  due  to 
the  perpetuation  of  requirements  that 
are  frequently  inapplicable;  the  second 
is  excessive  specifications  which  lists 
not  only  the  performance  requirements 
but  also  how  to  achieve  them.  There  has 
been  some  improvement  in  the  latter 
category  the  past  few  years,  but  when 
overspecification  appears,  it  robs  the 
program  of  the  experience  and  innovative¬ 
ness  of  the  contractor  by  limiting  the 
degree  of  freedom  in  his  design  approach. 
The  message  is:  Specify  the  end  result, 
not  how  to  attain  it! 

Another  deterrent  in  effectively  conduct¬ 
ing  development  programs  is  overmanage¬ 
ment  of  the  contractor.  This  practice 
is  sometimes  innocently  initiated  under 
justifiable  circumstances  when  a 
particularly  difficult  problem  arises, 
especially  at  the  leading  edge  of  the 
state-of-the-art,  legitimately  requiring 
additional  aid,  or  when  the  contractor 
is  ill-equipped  to  handle  an  unantici¬ 
pated  problem.  However,  from  this 
understandable  origin  has  sprung  a  more 
widespread  practice,  which  unfortunately 
has  all  the  earmarks  of  empire  building. 
The  sad  outcome  in  the  latter  case  is 
that  in  the  absence  of  real  problems, 
the  "overmanagement"  becomes  engaged  in 
overzealous  attention  to  details,  and 
the  writing  of  more  specifications,  all 
of  which  saddles  the  contractor  with 
using  precious  technical  time  to 
respond.  Dare  we  suggest  that  the 
Government  is  occasionally  overstaffed 
in  some  areas?  I  know  of  at  least  two 
instances  when  the  number  of  overmanagers 
exceeded  the  workers! 

In  closing,  I  would  like  to  summarize 
the  changes  in  procurement  practices 


that  1  believe  should  be  made  to  improve 
mission  success.  Among  the  more  signif¬ 
icant  are: 

First:  Purchase  insurance  in  the  form 
of  second  source  or  back-up  programs  for 
all  critical  subsystems  that  could  be 
show-stoppers  in  major  development 
contracts. 

Second:  Eliminate  the  paranoic  reaction 
to  reasonable  failures  that  occur  during 
development.  A  certain  amount  is  normal, 
healthy  and  necessary  to  place  a  con¬ 
fidence  level  on  the  product  and 
consequently  on  mission  assurance. 
Persistence  of  this  fear  can  only  lead 
to  a  deterrence  in  pushing  the  state- 
of-the-art  in  technology  and  relegate 
us  to  the  position  of  a  second-rate 
power. 

Thi rd:  Reduce  the  significance  of  bid 
cost  as  a  factor  in  awarding  development 
contracts  and  institute  additional  means 
to  evaluate  technical  competence  such  as 
the  recent  initiative  concerned  with  past 
performance. 

And  finally,  we  should  not  reduce  our 
efforts  to  modify  practices  and  require¬ 
ments  that  place  an  undue  burden  on  the 
contractor  and  deflect  his  attention  from 
more  critical  areas.  These  areas  include 
a  premature  commitment  to  the  "illities," 
overspecification  and  irrelevant  spec¬ 
ification  of  the  product,  and  excessive 
monitoring  of  the  contractor  during  the 
conduct  of  the  program. 

In  closing,  I  recognize  that  the  pro¬ 
curement  process  is  a  difficult  one  that 
has  evolved  over  many  years  of  practical 
experience,  but  still  requires  the 
exercise  of  judgment.  I  know  there  are 
retorts  to  each  of  the  criticisms  dis¬ 
cussed,  but  I  believe  that  improvement 
in  the  system  can  best  come  from 
uninhibited  open  discussions  of  this 
type. 
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ui'l  .Oirections  in  Craftwrrker 
r r.'j  i  n  i  ng 

Jnivies  P.  Mitchell 

■A  !ni  n  i  ctrator,  Pureau  of  Apprenticeship 
and  Training 


I.  hnological  change  has  been  with  us 
i’ico  the  discovery  of  fire  and  the 
irv.ntion  of  the  wheel.  In  1945, 

Arthur  Clarke,  the  author  of  "2001"  and 
■ifher  sc i ence-f i ct ion  works,  wrote  an 
irticie  suggesting  the  poss i b i I i ty  of 
j.'nchronous  s<5tellites.  In  1957  the 
Pur.ians  launched  the  first  satellite; 
lilt  i  ri  1962,  NASA  launched  the  first 
tr  irnniun  i  cat  i  ons  satellite,  Telstar. 

In  we  had  Arthur  Clarke's  first 

synchronous  orbit  satellite,  Syncom, 
ari  ability  essential  to  low  cost 
yjtellite  communications  of  today. 

ThoT'j  is  little  question  that  the  pace 
■c'  chiinae  has  quickened  in  the  last 
girirfc’r  of  this  century.  With  increas¬ 
ingly  complex  machines,  widespread 
grlication  of  computers,  modular 
'  il'r  in  ition  in  many  fields,  industrial 
ro[.  rt  ,,  sophisticated  information 
r  r'of  ('•/  ing,  electronic  teaching,  and 
siv/rf.'  in  an  almost  endless  list,  it  is 
..'oar  t h  1  (■  the  '"edesign  of  work  is 
i  m;.,!;.:  t  i  n  1  and  t  r.ins  form  i  ng  both  blue 
..oliar  I  :i  i  i  t  i.)ria  I  craft  and  production 
i)[v  ant  white  collar  and  professional 
,!  t  i  V  i  t  i .  'S  IS  well.  And,  it  is 
i  nr.r  ‘  ..r  ■  i  "  not  lessening,  dependence 
u[ion  I  ho  worker  who  has  mastered  the 
br  1  it  r.  io  or  fami  ly  of  related  task 
skill.;  that  make  up  a  "trade,"  who 
under,  lands  "why"  as  well  as  "how," 
who  d< 'mens fra tos  a  maturity  of  judgment 
that  is  the  ruark  of  a  craftworker  able 
t.  transl.jte  into  reality  the  vision  of 
th..  sciontist  and  design  of  the 
•  •ngi  ri'.'-r. 

i  fio  I  n,;.  I o/in. in  t  Out  I  ook  or  The  BLS  model 
:  for  Cra  ftworker  for  its 

employment 

I  r.)  ..'tions  uses  four  maj’or  groups  in 
whi.ti  .  iis:u(' a  t  ions  are  divided:  (I)  white 
I  ill  ir  Workers  in  professional,  techni- 
o  1 1  ,  lerii.al,  sales,  managerial  jobs; 

{.’)  hlue-oollar  workers  in  craft, 

:■  r  iiivo,  and  lab(5ring  jobs;  (3)  service 
w-tk'  r  ;  and  (4)  farm  workers.  White- 
oil  ir  w.  rl-.-r;,,  the  largest  group,  now 


represent  about  one-half  the  labor 
force,  cxceed'^d  in  number  the  blue- 
collar  group  in  1955,  and  tias  experi¬ 
enced  and  will  continue  to  experience 
the  hiqnest  growth  rate.  Blue-collar 
workers  have  grown  at  a  slow  rate, 
somewhat  below  that  for  service  workers, 
and  farm  workers  continue  a  decline 
beginning  with  the  I950's.  Table  I 
(See  attachment),  gives  employment  da  1a 
for  these  four  occupational  groups  for 
1 976  and  as  projected  for  1 985. 

Within  the  blue-collar  group,  craft- 
workers  will  increase  from  11,278,000 
in  1976  to  13,700,000  in  1985  a  per¬ 
centage  change  of  21.6^  which  compares 
with  19.2  %  change  for  all  groups. 
Construction  occupations,  mechanics 
and  repairers  will  account  for  about 
two-thirds  of  the  gain. 

Employment  growth  is  only  part  of  the 
demand  side  of  the  equation.  The  need 
to  replace  workers  who  die  or  retire 
is  expect'ud  to  produce  nearly  twice 
the  jobs  as  employment  alone.  In 
addition  to  work  separations,  the 
transfer  of  workers  between  occupations 
is  an  important  factor  in  occupational 
mix.  The  Bureau  of  Labor  St,stistics 
estimates  of  job  openings  from  growth 
and  replacement  for  the  four  major 
groups  is  shown  in  Table  2  (see 
attachment)  and  graphically  in 
Chart  I  (see  attachment).  The  net 
craftworker  demand  over  the  period, 
assuming  an  equal  annual  rate,  will 
range  between  500  to  600,000  openings 
annua  My. 

Supply  and  Training  Over  simplified, 
of  Craftworkers  the  potential 

supply  of  workers 

for  any  occupation  consists  of  persons 
already  emfiloyod  in  thjf  field  plus 
individuals  availible  from  other  sources. 
These  may  include  first  time  labor 
market  entrants,  persons  cr'mp  I et i  n.s 
training  programs  specific  to  the 
occupation,  completors  from  related 
fields,  qualified  persons  employed  and 
transferring,  unemployed,  i  mn  i  .;ran  t  , , 
and  persons  not  in  the  lab't-  force. 

For  many  occupafi(5ns  that  r'equir'  :•  ri-.il 
training,  adequate  analyses  t  surgdy 
are  impossil)le  since  only  limitet  data 
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on  training  completions  and  entry  rates 
are  ava i lable. 

Private  enterprise  engages  in  training 
and  education  to  meet  various  business 
needs:  (I)  to  train  new  employees, 

(2)  to  improve  employee  performance  in 
present  jobs,  and  (3)  to  upgrade  employ¬ 
ees  for  new  jobs  and  responsibilities 
including  replacement  needs. 

With  respect  to  skilled  occupations  the 
three  on- job  training  paths  are 
(I)  apprenticeship,  (2)  structured 
on-the-job  instruction,  and  (3)  learn¬ 
ing  by  observation  and  exposure  or 
doing.  Apart  from  a  limited  1975  mail 
survey  of  employer  training  in  about 
5,000  establishments  for  14  occupations 
in  4  metal  working  industries  conducted 
by  BLS,  no  comprehensive  data  on  train¬ 
ing  within  American  industry  is  avail¬ 
able.  The  limited  survey  indicated 
about  15?  of  the  establishments  provided 
structured  training  for  skills  with  a 
high  incidence  of  apprenticeship. 

Formal  programs  of  apprenticeship  recog¬ 
nized  by  the  Bureau  of  Apprenticeship 
and  Training  or  a  State  apprenticeship 
agency  are  an  important  source  of  new 
craftworkers  and  one  for  which  we  have 
data.  No  estimate  is  available  on 
apprenticeship  outside  the  system  which 
today  has  a  formal  enrollment  ("regis¬ 
tration")  of  more  than  300,000  appren¬ 
tices  in  over  800  occupations.  Last 
year  some  130,000  new  enrollments  or 
indentures  were  obtained  and  completions 
were  about  54,000.  Registered  appren¬ 
ticeship  is  highly  developed  in  the 
construction  industry,  yet  even  in  this 
field,  according  to  best  estimates, 
only  40?  of  the  skilled  construction 
workers  learned  their  job  through  formal 
training.  Predominately  in  the  private 
sector  and  for  occupations  in  the 
blue-collar  group,  formal  apprenticeship 
is  gradually  extending  its  base  to 
include  occupations  normally  found  in 
the  white-collar  category,  in  government 
and  in  the  uniformed  military  service. 
About  56?  of  registered  apprentices  are 
in  construction,  24?  in  manufacturing, 
and  the  balance  in  all  other  industries. 
Roughly  the  same  proportionate  distri¬ 
bution  prevai Is  with  respect  to 
comp  I et ions. 


It  is  quite  evident  that  formal  appren¬ 
ticeship  in  its  present  configuration, 
cannot  be  relied  upon  to  meet  BLS 
projected  new  craftworker  needs.  The 
annual  rate  of  500  to  600,000  such 
workers  needed  is  more  than  nine  times 
greater  than  the  current  annual  output 
from  the  system. 


Issues  and  New  As  a  form  of  craft 

Directions  training,  apprentice¬ 

ship  has  strong  con¬ 
ceptual  advantages.  By  its  users,  it 
is  held  to  be  the  least  costly,  most 
effective  method  for  the  development 
of  highly  skilled  workers  in  the  short¬ 
est  time  span.  Why  then  has  the  system 
failed  to  become  the  universally 
accepted  method  for  ski  I  led  worker 
development?  While  the  question  may  not 
be  satisfactorily  answered,  the  search 
reveals  probable  causes  and  raises 
issues  with  respect  to  development  of 
the  Nation's  skilled  worker  resources. 

Pub  lie  Policy  While  apprenticeship  is 
a  historic  institution, 
public  policy  in  appren¬ 
ticeship  is  of  relative  recent  origin. 
The  National  Apprenticeship  Act  is 
dated  August  1937,  and  was  proceeded  by 
specific  apprenticeship  legislation  in 
the  two  States  of  Oregon  and  Wisconsin. 
The  Federal  Act  simply  states  that  it 
is  in  the  public  interest  to  develop 
skilled  workers  through  apprenticeship, 
authorizes  the  Secretary  of  Labor  to 
establish  standards  for  the  welfare  of 
apprentices,  encourages  the  States  to 
take  similar  action,  and  authorizes 
appointment  of  advisory  committees.  An 
advisory  committee  was  appointed  and  it 
functions  today  as  the  Federal  Committee 
on  Apprenticeship.  Twenty-nine  States, 
District  of  Columbia,  Puerto  Rico  and 
the  Virgin  Islands  have  enacted  appren¬ 
ticeship  legislation.  About  one-half 
the  States  have  an  operating  agency 
with  staff. 

The  language  of  the  National  Act  is 
hortative.  The  program  is  advanced  by 
promotion  and  persuasion.  No  subsidy 
accompanies  registration.  There  are 
no  "grants- i n-a i d"  to  State  apprentice¬ 
ship  agencies. 


371 


Programs  of  apprenticeship  meeting 
standards  published  by  the  Secretary 
are  "registered"  either  directly  with 
BAT  in  21  States  or  the  State  agency  in 
the  remainder.  Registered  programs  and 
their  sponsors  are  subject  to  Equal 
Opportunity  Regulations,  29  CFR  30. 

Expanding  the  System  In  an  effort  to 

expand  the  program 
particularly  into 

industries  where  it  is  not  well  repre¬ 
sented,  a  number  of  initiatives  have 
teen  mounted  by  Secretary  Marshall. 

These  have  included: 

-  demonstration  projects  in  10  cities 
where  small  employers  are  banded 
together  with  government  financing 

a  program  coordinating  -  administra¬ 
tive  staff.  No  training  subsidies 
are  provided; 

-  demonstration  projects  in  7 
locations  to  directly  link  high 
school  seniors  prior  to  graduation 
into  an  apprenticeship  position.  A 
partial  training  cost  reimbursement 
is  given  the  sponsoring  employer; 

-  contracting  promotional  services  with 
national  organizations  of  business 
and  labor  for  the  organization  to 
engage  in  efforts  leading  to  estab¬ 
lishment  of  apprenticeship  among  its 
affiliate  local  units.  No  training 
subsidies  are  provided. 

In  addition  a  variety  of  efforts  at  the 
local  level  to  establish  program  link¬ 
ages  with  CETA  and  supportive  CETA 
financed  outreach  efforts  for  minorities 
and  women  have  been  undertaken. 

Given  the  voluntary  nature  of  the  U.S. 
apprenticeship  system,  the  options  for 
substantial  expansion  of  systematic 
skill  training  appear  to  be; 

I.  To  provide  financial  incentives 
partially  offsetting  training  costs 
to  induce  more  employers  to  engage 
in  training.  Most  employers  do  not 
view  an  investment  in  training  in 
the  same  manner  as  an  investment  in 
capital  goods.  Rather,  it  is  regarded 
as  a  high  risk  given  the  mobility  of 


workers  and  employers,  both  in  private 
and  public  sector,  will  generally 
elect  to  employ  hioher  skilled  and 
qualified  workers  when  availuDle. 
Experience  with  :>ofh  MDTA  and  CETA 
OJT  would  Indicate  an  employer's 
decision  not  to  train  cari  be 
reversed  through  subsidies.  The 
amount  and  form  are  debatable. 

2.  To  increase  substantially  the 
promotional  effort,  absent  incentives, 
and  the  number  of  people  engaged  in 
promoting  and  assisting  industry  to 
establish  programs.  This  does  no'* 
necessarily  mean  increasing  Federal 
staff;  it  suggests  increasing  State 
and  private  sector  staff,  the 
former  through  grants- i n-a i d ,  the 
latter  through  conti'acted  services. 

Early  in  IR'^R,  the  PAT  floated  for 
discussion  purposes  a  listing  of 
possible  apprenticeship  legislative 
specifications.  These  included; 

-  financial  support  to  State 
apprenticeship  agencies 

-  grants  to  apprenticeship  sponsors 

-  subsidies  in  critical  skill 
shortage  occupations 

-  public  work  for  unemployed 
apprentices 

-  tax  credits  for  apprenticeship 
tra i n i ng 

-  apprenticeship  in  the  Federal 
Government 

-  mandatory  apprenticeship  in  Federal 
contracts 

-  emergency/disaster  skill  worker 
cadres 

-  presentation  of  historic  hand  skills 

-  tripartite  industry  training 
counc i I s . 

Prospective  legislation  was  discussed 
with  leaders  in  the  apprenticeship 
community  from  both  management  and 
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labor,  with  the  Federal  Committee  on 
Apprenticeship,  with  the  American 
Apprenticeship  Round  Table,  and  various 
others.  The  response  could  be  charac¬ 
terized  as  far  less  than  enthusiastic 
and  it  is  clear  it  would  be  most  diffi¬ 
cult  to  obtain  industry  backing  for  any 
broad-based  apprenticeship  legislation 
in  the  near  future.  Shou I d  the  i ssue  of 
financial  support  be  surfaced  again? 

A  survivor  of  that  Christmas  Tree  of 
ideas  for  legislation  is  the  notion 
that  the  Government  should  be  a  leader 
in  the  employment  of  apprentices  wher¬ 
ever  appropriate.  The  8,000  odd 
appren+ices  in  civilian  employment  soon 
to  be  matched  by  an  equal  number  in  the 
military,  is  far  below  the  potential. 

We  anticipate  advancing  again  in  fiscal 
1982,  the  approach  of  an  Executive  Order 
requiring  apprenticeship  in  Federal 
employment  where  appropriate. 

This  has  the  element  of  mandatory 
apprenticeship,  the  second  issue  - 

Should  the  United  States  entertain 
the  Western  European  concept  of 
mandatory  apprenticeship? 

Our  one  experience  with  mandated  appren¬ 
ticeship  or  comparable  systematic  skill 
training  under  29  CFR  5a  during  the 
period  of  April  1971  to  revocation  of 
the  regulation  in  August  1975  does  not 
lend  support  to  a  system  of  compulsory 
apprenticeship.  While  limited  in  appli¬ 
cation  to  federally  assisted  construc¬ 
tion,  no  significant  impact  upon  the 
number  of  apprentices  could  be  observed. 

The  U.S.  educational  system  is  ill  pre¬ 
pared  to  follow  the  European  system  in 
preparing  students  for  entrance  into 
mandatory  apprenticeship.  Especially 
in  the  larger  cities,  the  secondary 
school  system  is  releasing  a  high  per¬ 
centage  of  young  men  and  women  that 
have  little  to  offer  in  the  job  market. 
The  CETA  system  of  remedial  training 
cannot  correct  the  deficiencies.  Indus¬ 
try,  in  a  mandatory  system,  would  be 
faced  unfairly  with  this  burden. 


Cone  I  us  ion  While  apprenticeship  and 
learning  by  doing  in  a 
structured  way  is  almost 
universally  regarded  as  the  most  eftfe- 
tive  and  efficient  method  of  developing 
the  skills  of  the  work  force,  the 
system  is  obviously  being  greatly 
underuf  i  I i zed . 

As  a  form  of  employment  as  well  as 
training,  it  depends  upon  a  job  avail¬ 
ability.  Any  major  expansion  in  the 
system  not  at  the  expense  of  another 
job  holding  worker,  requires  expansion 
in  the  economy. 

Apprenticeship  is  not  only  an  extensir'n 
of  education  for  youth  entrants,  it  is 
a  first  career  opportunity  for  many 
young  adults,  and  a  renewed  career 
opportunity  for  many  adults. 

No  true  measure  exists  as  to  what  the 
potential  for  apprenticeship  is. 
American  industry  and  Government  aliki.' 
have  a  training  capability  larnely 
untapped.  Government  policies  must  tu' 
framed  in  such  a  way  as  to  use  that 
capabiiity  while  at  the  same  time 
recognizing  the  vested  interest  of 
each  party:  I abor-management-the 
apprentice,  and  the  taxpayer,  in  the 
system. 
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TABLE  1 :  Employment  by  Major  Group 


Occupat i ona 1 

1976 

1985 

Z  Change 

No. 

No. 

% 

White  - 
Collar 

43,700 

49.9 

53,500 

51.3 

22.4 

Blue  - 
Collar 

1 

28,958 

33.1 

34,100 

32.7 

17.9 

Service 

Workers 

12,005 

13.7 

14,800 

14.2 

23.4 

Farm 

Workers 

2,822 

3.2 

1,900 

1.8 

-34.1 

Numbers  in  thousand 
Source:  BLS 


TABLE  2 


BiWlliRS  TO  PPOlllCTIVriY 
Max  Lehrer 

DIVISION  VICE  PRESIDENT,  GSD,  RCA 

Earlier  today,  Walt  Carrion  presented 
some  sobering  figures  demonstrating  that 
the  United  States  has  had  the  lowest  rate 
of  productivity  growth  of  the  11  major 
industrial  nations.  The  problem  is  actu¬ 
ally  worse  than  is  portrayed  by  the  com¬ 
parison  of  sixteen  year  averages  of  pro¬ 
ductivity  growth.  The  dismal  truth  is 
that  the  II.  S.  productivity  trends  have 
been  moving  in  the  wrong  direction  ever 
since  the  end  of  World  War  II. 

Here  is  a  comparison  that  has  been  made 
by  one  of  our  leading  economists- -Pierre 
Rinf  ret. 


PRODUCTIVITY  TRENDS  IN  THE 
UNITED  STATES 

19471979 

INCREASE  IN  OUTPUT  PER  MANUFACTURING  MAN  HOUR 


You  will  note  that  U.  S.  productivity 
increased  by  an  average  of  3.21  annually 
during  the  post  World-War  II  period  1947- 
1965.  During  the  Viet-Nam  War  Period, 
1966  through  1976,  the  annual  productiv¬ 
ity  increase  dropped  to  1.8°6.  From  1972 
to  1979,  it  dropped  again  to  0.81. 


PRODUCTIVITY  TRENDS  IN  THE 
UNITED  STATES 

19471979 

INCREASE  IN  OUTPUT  PER  MANUFACTURING  MAN  HOUR 


According  to  the  President's  Economic  Re¬ 
port  submitted  to  the  Congress  on  Janu¬ 
ary  31,  productivity  actually  decreased 
by  2.0%  during  the  year  1979.  Further- 
more,  the  Economic  Report  forecasts  an 
additional  decrease  of  0.5%  during  1980. 
There  are  many  who  feel  that  this  projec¬ 
tion  is  just  as  optimistic  as  the  already 
repudiated  projection  that  inflation 
would  only  amount  to  10.4%  in  1980. 

In  his  recent  second  videotape.  General 
Alton  D.  Slay.  Commander  of  the  Air  Force 
Systems  Command,  voiced  his  concern  over 
lagging  U.  S.  productivity  and  used  fig¬ 
ures  covering  a  different  time  frame  to 
make  his  point. 


INTERNATIONAL  PRODUCTIVITY  RANKING 

INCREASE  IN  OUTPUT  PER  EMPLOYEE  HOUR  IN  MANUFACTURING 
IS70IS7( 


AVERAGE  ANNUAL  GROWTH  IPERCENTI 

SOtKCf  U  S  Dtp!  9*  ItNoi 
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Based  on  these  figures.  General  Slay  con¬ 
cluded  that  the  U.  S,  would  be  surpassed 
in  overall  productivity  by  France,  Ger¬ 
many  and  Japan  in  just  a  few  years. 

WORKER  PRODUCTIVITY  PROJECTIONS 

IBXSEO  OH  PBOJECTEO  CIIOWTH  lUTES-AlL  WOUSmiESI 


WORKER  ^ 
PRODUCTIVITY 
ASA  100^^ 
PERCENT  OF 
U  S  (BASE 
YEAR  19771 


U  S  1  5%  PER  YEAR 


■  FRANCE  3.8% 


GERMANY  4.0% 

'u  s.  WORKER  PRODUCTiviTY 
'JAPAN  6  3%  OVERTAKEN: 

FRANCE  1986 
GERMANY  1987 
JAPAN  1988 


In  view  of  the  actual  decrease  in.  U.  S. 
productivity  experienced  in  1979  and  pro¬ 
jected  in  1980,  General  Slay's  projec¬ 
tions,  grim  as  they  are,  are  also  proba¬ 
bly  optimistic  concerning  the  time  re¬ 
maining  before  our  overall  productivity 
is  overtaken. 

The  threat  to  our  future  does  not  only 
arise  from  the  highly  industrialized  na¬ 
tions  of  the  world.  During  the  past  de¬ 
cade,  a  significant  portion  of  our  elec¬ 
tronic  components  and  systems  have  been 
produced  in  the  Far  East.  A  main  at¬ 
traction  has  been  the  plentiful  supply 
of  labor  available  at  low  wage  rates. 

The  average  monthly  manufacturing  wage 
in  the  United  States  today,  according  to 
the  Department  of  Labor,  is  $697.  Fringe 
benefits  add  some  30  to  351  to  our  labor 
costs.  Compare  this  with  average  monthly 
wages  of  $240  in  Hong  Kong,  $165  in  Sin¬ 
gapore,  $130  in  South  Korea  and  $110  in 
Taiwan. 
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Business  Week  magazine  forecasts  that  the 
trend  in  electronics  in  the  1980 's  will 
be  to  establish  new  plants  in  even  lower 
labor  cost  areas  such  as  China,  India, 
Indonesia,  Malaysia,  the  Philippines, 

Sri  Lanka  and  Tliailand.  At  the  same  time^ 
the  established  facilities  in  Hong  Kong, 
Singapore,  South  Korea  and  Taiwan  will  be 
shifting  from  labor-intensive  production 
to  capital-intensive  production.  It  is 
obvious  that  U.  S.  plants  will  have  a 
tremendous  challenge  in  conpeting  with 
foreign  plants  equipped  with  the  latest 
machinery  and  technology  and  employing  a 
labor  force  whose  cost  is  a  fraction  of 
that  of  the  United  States. 

The  pattern  of  increased  competition  from 
abroad  is  not  confined  to  electronics. 

Our  automobile  industry  is  reeling  from 
the  inroads  of  foreign  competition.  Our 
aerospace  industry  no  longer  has  a  free- 
world  monopoly  on  providing  satellites 
and  modem  aircraft.  Furthermore,  the 
pressures  generated  by  the  policy  to 
pursue  NATO  RSI  (Rationalization,  Stan¬ 
dardization  and  Interoperability)  will 
intensify  free-world  conpetition  for  all 
types  of  military  equipment  and  weapons. 
Obviously,  many  factors  affect  the  abili¬ 
ty  of  U.  S.  industry  to  produce  quality 
products  at  competitive  prices.  The  sin¬ 
gle  most  inportant  factor,  however,  is 
the  ability  and  willingness  of  industry 
to  invest  in  modem  production  and  test 
facilities. 

The  FY  1981  Department  of  Defense  Budget 
calls  for  an  increase  of  5.4  percent  in 
real  terms  (that  is,  adjusted  for  infla¬ 
tion)  and  projects  an  average  annual  in¬ 
crease  of  4.4  percent  during  the  next 
four  years.  The  Secretary  of  Defense  has 
observed  that  "the  percentage  of  our  GNP 
devoted  to  defense  has  fallen  from  8.6 
percent  to  5.0  percent  since  1962."  Ihe 
proposed  FY  1981  budget  would  increase 
the  Defense  share  of  GNP  from  5.0  percent 
to  5.2  percent.  Yet  tliere  is  serious 
qioestion  whether  this  modest  increase  can 
be  accomplished. 

Our  inability  to  accelerate  defense  pro¬ 
duction  arises  directly  from  tlic  fact 
that  we  have  allov^cd  our  industrial  base 
to  deteriorate.  Hie  problem  was  well  de¬ 
scribed  in  a  recent  analysis  made  by  Busi¬ 
ness  Week  Magazine,  entitlctl  'l\1iy  tlie  U. 

S.  Can't  Reami  last" 
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Nkinv’  comp;mies  once  bid  eagerly 
on  defense  contracts.  They  built 
their  resources  to  handle  them 
but  then  suffered  in  the  aerospace 
slump  of  the  early  1970 's  and  they 
Just  disappeared.  And  others  stead¬ 
fastly  refuse  to  expand.  They  are 
concerned.  .  .  .about  having  to  go 
out  and  make  capital  expenditures 
to  take  care  of  peaking  Government 
demand  and  tlien  having  Government 
say  'thanks'  and  walk  away. 

Subcontractors  also  corplain  about 
low  profits,  naive  procurement 
officers  and  heavy  paperwork. 

finally.  Business  Week  quotes  one  key 
aerospace  executive's  view  of  what  changes 
would  ha\e  to  be  made  to  entice  coTiipanies 
liack  into  the  defense  business. 

Of  key  importance  would  be  multi¬ 
year  procurement  budgeting  so  that 
contractors  could  plan  intel¬ 
ligently  their  own  capital  and 
manpower  investments.  Also  needed 
....  is  tax  reform  aimed  at 
allowing  more  liberal  writeoffs 
of  capital  investment.  With  sucli 
changes.  .  .  .  new  machine  tools, 
forging  presses  and  other  capital 
equipment  would  become  available 
for  defense  needs. 

I  agree  wholeheartedly  with  these  recom¬ 
mendations,  neither  of  which  is  new. 


Even  the  General  Accounting  Office,  in  a 
recent  report,  recommended  that  the  Con¬ 
gress  consider  multi-year  funding  to  pro¬ 
vide  stability  and  take  advantage  of 
more  economical  production  practices. 

The  main  obstacle  to  multi-year  funding 
has  been  the  reluctance  of  Congress  to 
abandon  any  of  its  detailed  annual  re- 
views--and  revisions--of  the  defense 
budget.  However,  a  prudent  businessman 
will  not  make  binding  long-range  finan¬ 
cial  commitments  in  the  face  of  program 
uncertainty.  This  problem  is  exacerbated 
when  uncertainty  is  compounded  by  contin¬ 
ued  double  digit  inflation. 

Going  beyond  statistics,  clearly  some¬ 
thing  is  very  wrong  when  defense  support¬ 
ing  industry  has  a  two -to -three  year 
backlog  and  steadily  increasing  leadtimes 
and  yet  does  not  invest  in  new  equipment. 

During  the  Korean  War,  provision  of  an 
investment  tax  credit  for  pi'ojects  certi¬ 
fied  by  the  Office  of  Emergency  Prepared¬ 
ness  was  a  simple  and  successful  device 
for  expanding  defense-related  production 
capabilities.  Today,  however,  neither  a 
tax  credit  nor  accelerated  depreciation 
alone  ivill  do  the  trick.  Today  we  have 
to  deal  with  the  costly  recordkeeping  and 
other  burdens  imposed  by  the  Cost  Ac- 


counting  Standard  409.  Since  the  Cost 
Accounting  Standards  Board  has  repeatedly 
asserted  that  its  only  concern  is  what  it 
considers  "good  accounting  practice," 
regardless  of  underlying  national  policy, 
any  legislative  steps  taken  to  provide 
shorter  depreciation  lives  could  be  un¬ 
done  unless  the  statute  directed  the  CAS 
Board  to  let  the  national  interest  pre¬ 
vail  over  alleged  "good  accounting  prac¬ 
tice." 

We  all  know  that  interest  rates  are  at 
the  highest  levels  in  U.  S.  History. 


Chase  Lifts  • 
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The  headlines  that  tell  us  that  the  prime 
rate  is  close  to  201  don't  tell  the 
whole  story.  When  a  conpany  borrows,  it 
cannot  obtain  all  the  money  covered  by 
the  loan.  Instead,  it  is  required  to 
maintain  a  compensating  balance  that 
generally  amounts  to  201  of  the  loan. 

This  means  that  a  201  nominal  interest 
rate  becomes  255.  Interest  is  an  even 
greater  burden  for  smaller  companies,  who 
generally  have  to  pay  several  points  over 
the  prime  rate. 

These  staggering  interest  costs  erode 
industry  profits  and  inhibit  the  invest¬ 
ment  in  both  production  facilities  and 
inventories  required  to  reverse  recent 
negative  trends  in  productivity.  When 
this  is  combined  with  the  long-term  un¬ 
certainties  and  increasing  risks  of  de¬ 
fense  procurement,  it  may  lead  many  con¬ 
tractors  to  devote  their  efforts  and  re¬ 
sources  to  more  profitable  commercial 
business. 

Some  of  the  remedies  to  correct  this  sit¬ 
uation  lie  within  the  control  of  the 


F.xecutive  Branch.  Other  remedies,  par¬ 
ticularly  the  authorization  of  multi-year 
procurement  and  the  provision  of  acceler¬ 
ated  depreciation  for  capital  investment, 
would  require  Congressional  action.  In 
a  Presidential  election  year,  the  pros¬ 
pects  are  not  very  bright  for  action  on 
some  of  the  fundamental  changes  needed. 
Yet  the  recent  events  in  Iran  and  Afghan¬ 
istan  clearly  demonstrate  the  urgent  need 
to  improve  our  readiness  and  capability 
to  respond  to  similar  challenges  to  our 
way  of  life. 
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PAST  PERFORMANCE  -  A  MAJOR  CONCERN 
IN  THE  AWARD  OF  CONTRACTS 

Bernard  L.  Weiss 

Colonel,  USAF 

DEPUTY  FOR  CONTRACTING  &  MANUFACTURING 

AERONAUTICAL  SYSTEMS  DIVISION  (AFSC) 

In  his  Policy  Letter  #  5,  dated  20  April 
1978,  General  Slay  expressed  his  policy 
that  special  emphasis  he  placed  on 
contractor's  performance  on  past  contracts 
as  part  of  the  source  selection  evalua¬ 
tion  process.  This  policy  was  reiterated 
in  the  well-known  Policy  Letter  //  22  of 
28  July  1978,  listing  specific  commer¬ 
cial  practices  to  be  emulated  by  Systems 
Command.  The  guidance  has  now  been 
established  in  the  more  enduring  form  of 
an  Air  Force  Systems  Command  Regulation, 
AFSCR  550-22,  dated  6  Dec  1978. 

General  Slay's  message  is  clear;  we  are 
to  find  a  way  to  award  Government 
business  to  manufacturers  we  can  expect 
will  deliver  a  quality  product,  Ca  time 
and  at  the  established  price.  With  the 
rising  costs  of  military  equipment,  the 
threatening  world  situation,  and  frequent 
evidence  of  taxpayer  dissatisfaction, 
past  performance  evaluation  must  become 
an  intrinsic  part  of  our  evaluation 
process. 

Evaluation  of  a  contractor's  past  per¬ 
formance  is  not  a  new  requirement.  The 
need  has  been  around  for  as  long  as  there 
has  been  an  opportunity  to  select  a 
single  contractor  from  two  or  more 
competing  sources  for  an  item  of  defense 
equipment.  The  Contracting  Officer  must 
determine  that  a  prospective  contractor 
is  responsible  before  he  can  award  a 
contract.  Previous  efforts  at  institut¬ 
ionalizing  the  evaluation  of  past  per¬ 
formance  have  not  met  with  overwhelming 
success,  however,  and  a  brief  review  of 
some  of  these  efforts  might  prove  useful 
as  a  backdrop  for  the  current  initiatives. 

From  the  late  1960s  through  the  early 
1970s,  a  number  of  evaluation  systems 
existed  to  provide  data  for  use  in 
source  selections.  Systems  Command,  for 
example,  started  a  program  in  1956  to 
collect  performance  reports  upon 
completion  of  research  and  development 
contracts  over  $25,000.  In  1963,  the 
Department  of  Defense  Implemented  the 


Contractor  Performance  Evaluation  (CPE) 
Program  which  required  periodic  reports 
during  performance  of  development  con¬ 
tracts  of  $2.M  or  more.  The  Contractor 
Performance  Record  (CPR)  Program  was  a 
formal  DOD  program  instituted  in  1968 
to  collect  data  on  supply  and  equipment 
contracts  valued  at  $100,000  or  more. 

The  CPR  program  also  required  reports 
upon  completion.  Working  groups  were 
formed  in  support  of  source  selections 
to  analyze  the  data  from  these  formal 
systems  and  from  other  official  DOD, 

Army,  Navy  and  DSA  records,  and  provide 
a  report  to  the  Source  Selection  Author¬ 
ity.  All  of  the  formal  systems  have 
been  out  of  existence  since  the  early 
1970s,  The  CPE  System  was  abolished 
in  November  1970  because  it  was  not 
considered  cost  effective,  and  the  rest 
followed  suit  shortly  thereafter.  Past 
performance  continued  to  be  a  consider¬ 
ation  in  the  award  of  all  contracts, 
but  without  any  formal  evaluation  system 
or  procedures  until  1978. 

In  July  1978,  a  decision  was  made  by 
Air  Force  Systems  Command  to  conduct  a 
test  of  the  use  of  past  performance  as 
a  source  selection  criterion.  This 
method  of  highlighting  the  importance  of 
our  experience  with  contractors  was  to 
be  tested  on  both  AFR  70-15  (major) 
source  selections,  and  AFSCR  80-15  (R&D) 
source  selections.  The  test  was  expected 
to  determine: 

a.  Whether  use  of  past  performance 
as  a  major  ranked  area  can  be  expected 
to  produce  the  desired  result  of 
eliminating  an  offeror  with  unsatisfact¬ 
ory  performance. 

b.  Whether  use  of  past  performance 
as  a  general  consideration  is  adequate 
to  produce  that  same  result. 

c.  Whether  use  of  past  performance 
in  source  selection  in  either  manner  is 
beneficial  in  terms  of  administrative 
complexity,  lead  time,  work  load,  and 
cost  impacts. 

d.  Whether  use  of  past  performance 
should  be  a  matter  of  predetermination 
at  the  Source  Selection  Authority  (SSA) 
or  equivalent  level. 
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Fourteen  Systems  Command  programs  were 
selected  for  this  test,  representing 
source  selections  covering  all  phases  of 
the  acquisition  process,  from  concept 
validation  to  production.  Several  sub¬ 
stitutions  had  to  be  made  before  the 
test  was  over,  and  one  program  cancella¬ 
tion  came  too  late  for  a  substitution 
within  the  allotted  time.  Not  all 
selections  were  completed  within  the 
time,  but  the  data  base  that  was  obtained 
was  considered  adequate  for  test  purposes. 
Dollar  value  of  the  programs  varied  from 
$200,000  to  over  $100. M.  Three  were 
"fly-off"  competitions,  two  were  full- 
scale  development,  eight,  research  and 
development,  and  one  was  for  a  valida¬ 
tion  phase.  Contract  type  Included  four 
each  CPFF  and  FPI,  and  six  firm  fixed 
price. 

The  results  of  the  test  which  was 
completed  in  late  summer  1979  indicated 
that  evaluators  want  to  use  past  per¬ 
formance,  and  that  offerors  want  it  used 
as  well.  There  was  agreement  that,  if 
some  of  the  problems  can  be  overcome, 
past  performance  has  great  potential. 

It  would  bring  to  the  front  of  the 
evaluation  our  experience  with  the 
offeror,  add  to  (or  detract  from)  our 
confidence  in  his  technical  and  schedule 
and  price  promises,  and  serve  as  a 
motivator  during  performance  of  current 
contracts. 

The  most  difficult  of  the  problems  to  be 
overcome  is  the  question  of  devising  a 
system  that  would  be  fair  and  objective, 
without  reverting  to  the  data  bank  of 
previous  evaluation  systems.  The  data 
banks  didn’t  work.  Contractors  rated 
"excellent"  accepted  their  ratings; 
others  generated  what  General  Stansberry 
has  referred  to  as  a  "paper  war",  from 
which  neither  party  emerged  victorious. 
Another  problem  arises  in  evaluating  an 
offeror  without  relevant  past  performance, 
but  this  can  be  resolved  equitably  by 
using  a  neutral  rating.  We  must  pay 
attention  to  the  way  we  define  what  is 
relevant,  because  that  definition,  in  a 
given  source  selection,  will  establish 
the  kind  and  extent  of  past  performance 
data  to  be  evaluated.  The  effort  taken 
to  resolve  these  problems  and  create  a 
viable,  credible  system  for  using  past 
performance  will  be  repaid  in  higher 
quality  systems  and  equipment. 


Systems  Command  has  provided,  as  a 
modification  to  AFR  70-15,  some  def¬ 
inition  of  "relevant  past  performance." 
This  definition  leaves  room  for  Judg¬ 
ment,  and  will  evolve  as  we  gather  ex¬ 
perience  with  the  new  policy.  For  now, 
we  will  define  relevant  experience  as 
experience  which  is; 

"work  comparable  to  the  Instant 
acquisition,  completed  in 
accordance  with  a  contract, 
recently." 

It  would  Include,  for  example,  work  in 
the  same  or  a  similar  acquisition  phase, 
for  a  similar  item,  or  item  using 
similar  technology. 

When  General  Slay  refers  to  the  use  of 
past  performance  as  a  commercial  prac¬ 
tice,  he  is  on  target.  Past  perform¬ 
ance  is  always  influential  in  the  world 
of  lawnmowers,  automobiles  and  jet 
transports,  and  it  is  often  an  over¬ 
riding  factor.  Vendor  rating  systems 
are  a  vital  part  of  the  overall  business 
strategy  of  firms.  The  systems  in  use 
vary  in  sophistication  with  the  complex¬ 
ity  of  equipment  and  amount  of  sub¬ 
contracting  involved,  but  they  all  serve 
to  exclude  the  unacceptable  vendor,  and 
flag  the  questionable  one  so  that  suit¬ 
able  controls  can  be  imposed. 

It  is  not  realistic  to  assume  that  we 
in  Government  will  be  able  to  use  the 
same  basis  for  judgment  that  a  commercial 
firm  can  use  because  of  the  emphasis 
placed  on  awarding  to  the  low  bidder, 
and  the  fact  that,  in  a  protest,  the 
burden  of  proof  would  be  on  the 
Government.  Every  low  bidder  has  one 
Congressman,  two  Senators,  a  "get  well" 
program  and  500  reasons  why  their  poor 
past  performance  was  the  Government's 
fault  or  reasons  beyond  their  control. 
Compounding  the  problem  is  the  fact  that 
some  of  those  500  reasons  will  be  valid. 
We  can,  however,  do  a  better  job  through 
effective  evaluation  programs,  with 
positive  as  well  as  negative  aspects, 
with  continued  emphasis  during  source 
selection,  and  with  continued  study  to 
determine  where  changes  in  law  or  reg¬ 
ulation  are  needed  for  improved 
efficiency  of  our  source  selection 
process. 


The  current  Air  Force  Systems  Command 
policy  regarding  use  of  past  performance 
In  source  selection  is  stated  in  interim 
message  changes  to  Air  Force  Regulation 
70-15  and  Air  Force  Systems  Command 
Regulation  80-5,  dated  1  November  1979. 
These  changes  state: 

a.  That  past  performance  will  be 
established  as  a  specific  evaluation 
criterion  against  which  each  element 
of  the  requirement  will  be  measured. 

This  means  that  in  each  "area"  (such  as 
technical,  management,  or  cost),  there 
must  be  an  "item"  called  past  perform¬ 
ance.  As  an  alternative,  each  "item" 
(such  as  hydraulics,  electrical,  etc., 
under  technical)  could  have  a  past 
performance  factor. 

b.  That  the  past  performance 
criterion  shall  be  equal  in  ranking  or 
stature  to  all  other  criteria  if  all  are 
equal,  or  first  if  ranked  in  order  of 
importance.  This  means  simply  that  no 
item,  or  factor,  can  be  ranked  ahead  of 
past  performance.  It  can  be  equal,  or 
first,  but  never  subordinate  to  the 
other  ranked  factors. 

In  addition,  the  regulation  changes 
provide  for  verification  of  the  con¬ 
tractor's  submission,  and  for  Inclusion 
of  past  performance  data  in  all 
documents,  reports  and  presentations. 
Verification  will  be  accomplished 
primarily  by  the  cognizant  plant 
administration  activity,  but  this 
information  may  be  augmented  during 
the  course  of  other  source  selection 
related  plant  visits  such  as 
manufacturing  or  other  pre-award 
surveys. 

The  policy  is  clear.  The  question  is, 
can  we  make  it  work?  Can  we  come  up 
with  an  evaluation  methodology  that  will 
exclude  the  marginal  performer  from 
future  awards  and  still  conduct  our 
business  in  an  orderly  manner?  We 
think  the  answer  is  yes. 

The  emphasis  Imposed  by  the  changes  in 
our  regulations  are  a  step  in  the 
right  direction.  Accompanied  by  a 
positive  attitude  and  determination 
on  the  part  of  management  and  a  positive 
attitude  and  open  mind  on  the  part 


of  the  evaluators,  the  regulatory 
changes  will  work.  In  addition  to  these 
changes,  continued  emphasis  on  Product 
Assurance,  including  formal  recognition 
of  producers  building  quality  products, 
will  help  foster  the  attitude  that  the 
Air  Force  is  going  to  continue  to 
insist  on  contract  compliance,  and  is 
willing  to  reward  the  manufacturer  that 
does  a  superior  Job.  On  the  other  hand, 
a  low  rating  on  past  performance  in 
source  selections  should  be  Just  one  of 
the  sanctions  we  must  Impose  on  the 
marginal  producer  to  establish  credi¬ 
bility  on  our  part  as  to  our  demands  for 
quality  products.  We  must  also  make 
more  effective  use  of  existing  procedures 
such  as  mandatory  inspections  and  with¬ 
holding  of  progress  payments.  Deferring 
acceptance  until  corrections  are  made 
rather  than  use  of  "conditional" 
acceptance  is  another  seldom  Invoked 
technique.  We  have  made  some  headway 
In  these  Initiatives  over  the  past  two 
years  as  a  result  of  the  Quality 
Horizons  Study  sponsored  by  AFSC  looking 
for  ways  to  enhance  product  quality  of 
military  hardware,  and  other  efforts, 
but  we  still  have  a  long  way  to  go. 

The  evaluation  of  relevant  past  perform¬ 
ance,  even  if  it  was  not  used  to  exclude 
certain  manufacturers  through  use  of 
a  list  of  "ineligible"  bidders,  would 
still  be  of  value  to  the  Program  Office. 
For  example,  it  should  lead  us  toward 
more  logical  contract  administration 
procedures.  We  need  to  create  the 
flexibility  in  our  own  operations  to  take 
advantage  of  the  Information  the  past 
performance  evaluation  will  give  us,  and 
Increase  inspection  where  indicated,  for 
example,  and  also  decrease  surveillance 
where  indicated.  This  kind  of  flexi¬ 
bility  will  be  an  offshoot  of  the  past 
performance  evaluation  effort. 

In  summary,  we  have  made  a  good  start 
at  institutionalizing  Past  Performance 
In  source  selection.  While  we  recognize 
the  differences  in  business  environment 
between  the  Government  world  and  the 
commercial  world,  we  are  determined  to 
come  as  close  as  possible  to  the 
commercial  world  when  it  comes  to 
past  performance.  Research  is  continuing 
in  this  area,  to  help  identify  and  use 
pertinent  factors  that  constitute  past 


performance  information,  to  establish 
methods  to  assess  such  information, 
and  to  devise  procedures  to  effectively 
acquire  the  information  when  it  is 
needed.  Emphasis  will  continue  to  be 
placed  on  quality  performance  in  our 
contracting  and  manufacturing 
initiatives.  Past  performance  has 
joined  death,  taxes  and  Inflation  on 
the  list  of  things  that  are  "inevitable." 
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CONTRACTOR  ASSESSMENT 
DURING  PROGRAM,  SURVEYS 
DAVID  J.  AUSTIN  (MAJOR,  USAF) 
MGR.,  QA,  SPACE  SHUTTLE 
HQ  NASA 


Audits,  Reviews  and/or  surveys 
are  a  necessary  "evil"  that  we 
must  live  with.  In  many  programs, 
these  evils  are  neglected  or 
minimized  during  the  early  phases 
of  a  programs  life.  They  then 
tend  to  become  corrective  or 
punitive  in  nature  rather  than 
preventive.  They  are  often  call¬ 
ed  for  after  the  fact,  when  a 
failure  results  in  life  or  mission 
loss  or  cost  or  schedule  over 
runs.  We  (NASA)  prefer  to  be 
preventive  and  resolve  potential 
problems  before  they  occur  rather 
than  play  the  "fire  fighter"  role. 
Its  obviously  too  late  to  fix  a 
problem  or  system  after  a  space 
system  or  missile  failure  has 
occurred.  Until  we  get  a  Space 
Shuttle  into  orbit,  space  system 
failures  will  result  in  loss  of 
both  dollars  and  acquisition  of 
needed  intelligence,  weather, 
astrological,  etc.,  data.  After 
we  have  a  viable  Shuttle  Program 
some  satellites  will  be  retriev¬ 
able  and  repairable,  but  others 
and  missiles  will  not  be.  There¬ 
fore,  preventive  type  SR^QA 
surveys  and  reviews  will  be  the 
norm  rather  than  the  exception. 

In  NASA,  we  have  several  require¬ 
ments  or  directive  documents  that 
provide  guidance  or  define  the 
use  of  surveys,  audits  and  re¬ 
views.  Generally,  in  the  case  of 
HQS  conducted  survey,  these  will 
be  combined  into  one  periodic 
Safety,  Reliability,  Quality 
Assurance  and  Occupational  Health 
Review.  The  directives  also 
provide  for  reviews  at  the  Centers 
or  other  NASA  Organizations, 
Contract  Administration  Office, 
and  Contractor  Facilities. 


Today  I  will  discuss  these  reviews 
in  conjunction  with  the  Space 
Shuttle  Program,  but  other  NASA 
Programs  in  general,  require 
similar  reviews. 

The  highest  level  survey.  Level  I 
in  the  Space  Shuttle  Program, 
consists  of  a  Bi-Annual  Headquar¬ 
ters  Review  of  Level  II.  Level  II 
is  at  Johnson  Space  Center,  This 
survey  is  primarily  to  review  their 
compliance  to  Headquarters  SR5QA 
and  OH  Directives.  The  next  order 
is  a  Joint  Level  I  and  Level  II 
Review  of  Level  III.  Level  III 
Organizations  are  the  major  sub¬ 
systems;  External  Tank,  Orbiter, 
SSME  and  Solid  Rocket  Booster 
Project  Offices.  These  surveys 
are  to  assess  the  subsystem  pro¬ 
ject  offices  implementation  of 
Level  II  requirements  and  also  to 
review  their  surveillance  over 
prime  and  sub-tier  contractors. 

The  NASA  Centers  perform  SR§QA 
Surveys  at  the  Prime  Contractors 
on  an  annual  basis.  These  surveys 
are  fairly  in-depth  and,  to  some 
extent,  also  review  the  Government 
Resident  Contract  Administration 
Office  Assurance  Program.  At  this 
time  the  primes  are  also  reviewed 
as  to  their  surveys  and  controls 
over  vendors/subcontractors. 

Surveys  performed  at  sub-tier 
vendors  or  subcontractors  where 
more  than  one  prime  or  NASA  Center 
has  an  interest  fall  under  the 
Joint  Survey  Program  which  is 
managed  by  Level  II  at  JSC.  On 
prior  NASA  Programs,  experience 
has  shown  that  suppliers  manufac¬ 
turing  items  for  several  major 
subcontractors  involved  on  the 
same  program  have  often  been 
surveyed  by  each  customer.  In 
order  to  reduce  this  activity  to 
a  minimum  on  the  Space  Shuttle 
Program,  a  survey  committee  con¬ 
sisting  of  representatives  of  all 
the  major  element  contractors  has 
been  established  to  plan,  coordi¬ 
nate,  and  schedule  surveys. 

Survey  teams  have  been  organized 
of  all  the  involved  customers 
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including  the  NASA  Centers  so  that 
each  supplier  survey  can  be  com¬ 
pleted  at  the  same  time.  These 
surveys  are  scheduled  annually  or 
more  or  less  often  as  required. 

All  of  the  previously  mentioned 
surveys  tend  to  focus  on  large 
organizations  where  both  dollars 
and  resources  are  available  for 
controlling  the  product.  However, 
during  the  past  several  years 
there  has  been  a  significant 
decline  in  Government  SR§QA  empha¬ 
sis  at  the  lower  tier  levels  where 
raw/basic  materials  and  nuts  and 
bolts  products  are  produced.  I 
believe  that  there  has  been  an 
equal  decline  in  Prime  Contractor 
Surveillance  at  these  levels  also. 
We  could  cite  many  DOD,  NASA  and 
even  FAA  examples,  but  would 
rather  not  embarrass  anyone  here 
today.  It  is  important,  toward 
mission  assurance,  to  recognize 
these  short  falls  and  discuss  means 
of  correcting  the  problem  today. 

As  an  opener,  I  submit  that  the 
following  is  necessary  by  the 
Primes : 

1.  Identify  in  his  purchase 
order  the  proper,  realistic 
and  adequate  requirements 
for  supplier  performance. 

2.  Select  only  those  suppliers 
or  vendors  who  have  a 
history  of  providing  accept¬ 
able  products . 

3.  Review  the  potential  vendor/ 
supplier  prior  to  award, 
during  the  acquisition  pe¬ 
riod  and  maintain  receiving 
inspection  and  post  accep¬ 
tance  records  of  deficien¬ 
cies. 

4.  Require  vendors/suppliers 
to  take  corrective  action 
and  follow-up  for  effective¬ 
ness  . 

By  the  Government: 

1.  Where  we  the  Government  have 
relaxed  our  surveillance 


efforts,  request  for  one 
time  selective  surveys  of 
the  lower(est)  tier  vendors 
by  DCAS  should  be  consider¬ 
ed  . 

2.  In  lieu  of  the  above,  uti¬ 
lization  of  the  Joint 
Survey  Program  with  minimal 
team  members  may  be  desir¬ 
able  . 

Program  success  depends  on  com¬ 
plete  control  by  the  Prime  Con¬ 
tractors  over  all  vendors/subcon - 
tractors.  A  comprehensive  sur¬ 
veillance  program  should  go  a  long 
way  in  preventing  non-conforming 
materials  from  entering  end  items. 
Associated  with  the  Prime  Surveil¬ 
lance  Programs  the  Government 
Source  Inspection  System  will  add 
additional  assurance,  but  should 
be  applied  on  a  selective  basis  as 
necessary . 

Now  that  I've  given  you  my  thou¬ 
ghts  on  the  need  for  greater 
vertical  depth  in  the  area  of  QA 
surveys.  What  are  your  collective 
ideas? 


INCREASING  QUALITY  ASSURANCE  EFFECTIVENESS 

John  F.  Rytlewski 

QUALITY  SYSTEM  REVIEW  MONITOR, 

OFFICE  OF  THE  QA  DIRECTORATE, 

DCASR,  Los  Angeles 


The  Quality  System  Review,  (QSR),  a  De¬ 
fense  Logistics  Agency  program  to  evalu¬ 
ate  a  contractor's  Quality  Assurance  Sys¬ 
tem,  has  indicated  that  several  major 
deficiencies  are  common  to  most  problem 
contractors.  These  deficiencies  generate 
a  large  amount  of  corrective  action  time 
which  is  costly  in  terms  of  both  money 
and  time.  Most  of  these  deficiencies 
can  be  corrected  at  little  cost  to  the 
contractor  and,  as  a  by-product,  result 
in  a  higher  quality  product.  Before  I 
begin  a  discussion  of  these  common  defi¬ 
ciencies,  and  how  to  correct  or  prevent 
them,  I  should  briefly  describe  the 
Quality  System  Review  Program. 

My  primary  function  in  DCASR,  LA,  is 
managing  the  QSR  Program,  a  program  by 
which  the  Directorate  staff  independently 
evaluates  a  contractor's  Quality  Assurance 
System.  Generally,  contractor  candidates 
for  the  QSR  program  are  selected  because 
of  problems  relating  to  product  quality. 
Therefore,  in  most  instances,  QSRs  are 
performed  in  facilities  which  are  sus¬ 
pect.  Contractor's  staff  personnel  are 
told  at  the  entrance  briefing  to  make 
use  of  our  audit  for  their  own  benefit; 
use  it  as  a  free  consulting  service,  do 
not  fear  criticism  or  deficiencies.  The 
audit  team  members  are  relatively  un¬ 
biased  and  are  experienced  in  review 
methods.  All  deficiencies  are  thoroughly 
discussed  by  team  members  and  cognizant 
contractor  personnel  . 

We  have  found,  particularly  in  the  small 
operations,  that  management  is  unaware  of 
what  constitutes  a  good  Quality  System. 

We  cannot  tell  the  contractor  how  to  run 
his  facility,  but  we  can  and  do  make 
recommendations  concerning  methods  to 
effect  action  which  can  correct  noted 
deficiencies  and  preclude  similar  in¬ 
adequacies  . 

Although  most  of  the  problems  can  be 
corrected  at  little  or  no  cost,  it  is 
surprising  how  upper  management  refuses 


to  acknowledge  that  there  are  problems, 
problems  which  can  be  rectified  solely 
by  changes  in  policy.  This  is  especially 
true  when  management  has  been  in  the 
same  position  for  a  number  of  years  and 
is  responsible  for  the  present  inadequate 
policies.  The  following  examples  demon¬ 
strate  several  common  problems  and 
possible  corrective  actions: 

Lack  of  QA  Authority;  We  have  found 
that  in  many  facilities,  especially  the 
smaller  ones,  the  QA  Manager  is  merely 
a  figurehead,  and  manages  solely  as  a 
supervisor.  He  has  no  authority  to  en¬ 
sure  that  the  QA  System  is  an  effective 
tool  in  controlling  product  quality 
throughout  the  facility!  Quality  Assur¬ 
ance,  in  this  enviornment,  acts  only  as 
an  inspection  department  to  screen  out 
defective  material.  In  this  type  of 
operations,  there  is  usually  an  excessive 
amount  of  defective  material  submitted 
to  Material  Review,  due  to  the  lack  of 
controls  during  the  various  manufacturing 
processes.  Manufacturing  or  engineering 
is  usually  in  charge  and  the  Quality 
Assurance  department  exists  only  because 
of  contractual  requirements.  Corrective 
action  in  this  instance  must  be  initiated 
at  higher  management  levels.  QA  must  be 
allowed  to  function  as  a  seperate  divi¬ 
sion,  with  the  ability  and  responsibility 
to  initiate,  coordinate,  and  complete 
corrective  actions  throughout  the 
facility  whenever  warranted. 

Ineffective  Quality  Assurance  Organiza¬ 
tion:  As  a  general  rule,  the  Quality 
Assurance  Manager  must  be  a  dynamic  and 
forceful  individual.  He  must  act  as  a 
coordinator  between  the  various  divisions 
so  that  corrective  actions  requiring 
joint  efforts  are  effectively  imple¬ 
mented.  An  ineffective  QA  organization 
will  usually  be  recognized  by  a  record 
of  ongoing  similar  deficiencies.  Inspec¬ 
tion  efforts  will  eventually  become 
indifferent  because  of  a  lack  of  a  follow 
through  on  corrective  actions.  In  this 
example,  a  revitalization  of  the  quality 
effort  is  necessary.  Training,  motiva¬ 
tion,  and  changes  of  job  duties  or 
responsibilities  is  in  order. 

Submission  of  Defective  Material  to 
Inspection  by  Manufacturing:  It  may 
seem  odd  to  say  that  defective  material 
is  being  submitted  to  Inspection;  you 


might  ask.  -  isn't  that  why  inspection  is 
there  -  to  identify  deficiencies?  In¬ 
spection  is  there  to  ensure  a  quality 
product;  not  to  assist  in  a  manufactur¬ 
ing  workmanship  training  program,  or  to 
screen  items  submitted  by  obviously 
unmotivated  or  inattentive  personnel. 

First  line  manufacturing  supervisors  are 
the  key  to  manufacturing  a  quality 
product.  They  are  the  personnel  who 
must  bear  the  brunt  of  responsibility 
for  submitting  the  best  possible  product 
to  inspection.  Manufacturing  personnel 
will  build  only  to  the  quality  level 
required  by  their  supervisor.  First  line 
supervision  must  insist  on  the  best 
possible  quality  from  their  assembly 
workers;  the  statement,  "good  enough," 
is  not  acceptable  in  a  Quality  enviorn- 
ment.  Manufacturing  supervision  must  be 
trained  and  indoctrinated  with  the  con¬ 
cept  that  they  are  responsible  for  the 
quality  of  the  products  under  their 
cognizance.  They  are  responsible  for  en¬ 
suring  that  their  personnel  receive 
adequate  instruction  and  training  to 
accomplish  their  assigned  functions. 

Inaccurate  Purchasing  Data:  Purchasing 
must  pass  on  all  appropriate  contractual 
requirements  to  their  vendors.  Lack  of 
required  tests  and  data  for  vendor 
supplied  parts  is  just  as  defective  as  a 
missing  component.  Purchasing  and  the 
QA  organization  must  work  together  to 
ensure  that  all  applicable  contractual 
requriements  are  contained  in  sub-con¬ 
tracts  and  purchase  orders.  In  addition, 
suppliers  who  do  not  comply  with  all 
stated  requirements  must  be  compelled 
to  take  corrective  action  and  ensure 
that  similar  deficiencies  will  not  recur. 
You  are  paying  for  required  tests  and 
data;  make  sure  you  get  them. 

Lack  of  Vendor  Rating  System:  Lack  of 
vendor  data  by  which  you  can  readily 
anticipate  vendor  product  quality  can 
substantially  increase  your  incoming 
defective  material .  Unless  procedures 
are  established  by  which  you  can 
identify  problem  vendors.  Purchasing  will 
continue  to  place  additional  orders  with 
vendors  who  have  proven  to  be  inadequate. 
Problem  vendors  must  initiate  a  program 
of  immediate  corrective  action,  or  be 
replaced  with  more  reliable  suppliers. 


Hopefully  discussion  of  the  preceding 
problem  areas  will  cause  you  to 
re-examine  your  company's  current  policies. 
Correction  of  these  types  of  problems 
will  save  you  both  time  and  money,  and 
help  you  to  produce  a  quality  product. 


PERSONAL  PRIDE  IN  CRAFTMANSHIP 
Norman  S.  Wilson 

DIRECTOR,  GROUP  OPERATIONS 
SPACE  SYSTEMS  GROUP 
ROCKWELL  INTERNATIONAL 

Engineering  ingenuity  and  pride  in  personal 
craftmansliip  have  contributed  heavily  to  America 
achieving  world  leadership  in  industry.  Today, 
Americans  continue  to  display  a  wealth  of  engi¬ 
neering  ingenuity  in  a  world  of  mind-bc^ling 
teclinology.  But  what  about  pride  in  personal 
craftsmanship?  It  is  not  seen  as  readily  and  in  some 
areas,  craftsmanship  has  deteriorated.  Why?  Have 
people  really  stopped  taking  pride  in  their  work?  I  do 
not  think  so. 

A  few  years  ago,  there  was  a  great  personal  pride  in 
craftsmanship  throughout  the  work  force.  For 
example,  once  in  Rockwell’s  sheet  metal  department, 
the  location  of  two  holes  in  150  small  electrical 
junction  boxes  iiad  been  interchanged.  Inspection 
personnel  caught  the  error  and  returned  the  parts  to 
Manufacturing  to  be  corrected.  Four  mechanics  came 
to  work  the  next  Saturday  and,  without  punching  the 
time  clock,  repaired  the  boxes.  On  the  following 
Monday,  the  parts  were  accepted  by  Inspection. 

Why  did  four  people  come  to  work  on  one  of  their 
days  off  to  correct  the  mistake?  No  one  would  have 
been  fired  or  disciplined  for  this  error,  but  pride  in 
their  personal  craftsmanship  would  not  permit  them 
to  allow  the  boxes  to  remain  unacceptable  for  use. 

Another  example  is  a  loftsman  who  was  making 
a  template  for  a  complex  shaped  fairing.  As  he  was 
trimming  off  the  excess  material,  he  set  the  shear  to 
the  wrong  line  and  cut  off  part  of  his  layout.  Without 
a  word,  the  man  put  on  his  coat,  picked  up  his  lunch 
box,  punched  out  on  the  time  clock,  and  went  home. 
Why?  Making  that  mistake  was  such  a  blow  to  the 
man's  pride  that  he  became  physically  ill.  When  he 
came  to  work  the  next  morning,  his  layout  had  been 
spliced  together  so  that  it  was  as  good  as  new,  and  no 
one  mentioned  the  incident. 

Further  examples  include  power  brake  operators 
who  locked  their  machines  into  the  ON  position  to 
eliminate  the  time  wasted  by  the  repetitive  ON-OFF 
mode  of  operation.  Even  though  this  was  dangero  s. 


the  operators  did  it  because  it  almost  doubled 
production.  Safety  regulations  put  a  stop  to  this 
practice,  but  it  does  further  illustrate  personal  pride 
in  craftsmanship. 

Hydropress  operators,  in  an  effort  to  reduce 
tooling  costs,  experimented  on  their  own  time  with 
form  dies  made  from  various  materials.  They 
discovered  that  a  hard  wood  die  would  form  a  large 
quantity  of  parts  at  a  fraction  of  the  cost  of  the 
conventional  steel  die.  And  manufacturing  engineers 
have  worked  hours  of  unpaid  overtime  in  developing 
new  manufacturing  technologies  because  of  personal 
pride  in  their  craftsmanship. 

There  is  no  doubt  that  Americans  have  taken  great 
personal  pride  in  their  craftsmanship.  Today, 
however,  any  personal  pride  in  one’s  work  is 
overshadowed  by  examples  of  declining  productivity, 
poor  workmanship,  and  an  “1  don’t  care”  attitude. 
What  has  happened?  What  can  be  done  about  it? 

Many  factors  have  contributed  to  the  decline  of 
pride  in  craftsmanship.  The  change  in  the  moral 
attitude  of  the  country  is  certainly  a  major  factor, 
but  that  subject  is  better  handled  by  experts  in 
behavioral  science.  Factors  directly  related  to 
business  are  what  will  be  addressed  here. 

First,  craftsmanship  has  been  stifled  by  too  much 
direction  and  control.  Because  of  the  critical 
requirements  of  today’s  hardware,  the  increasing 
tendency  has  been  to  identify,  through  detailed 
step-by-step  instruction,  how  a  part  should  be  made. 
Admittedly,  this  is  necessary  in  some  cases,  but  in 
many  instances,  it  is  not  and  it  tends  to  be 
counter-productive.  Stcjvby-step  instruction  is  costly 
and  it  restricts  craftsmanship.  How  much  pride  can  a 
person  have  in  his  work  when  he  is  being  told  exactly 
how  to  do  it?  The  inference  is  that  the  mechanic  is 
not  smart  enough  to  do  the  job,  and  pride  in 
craftsmanship  will  not  flourish  under  these  condi¬ 
tions.  Part  requirements  should  be  identified  and  to 
the  extent  that  is  practical,  the  “how-to”  left  to  the 
mechanic.  The  result  will  be  increased  production, 
reduced  costs,  and  vastly  improved  morale. 

The  second  problem  is  that  the  image  of  the  work 
force  has  been  demeaned.  This  has  not  been  the  result 
ol  deliberate  intent  but  gradual  erosion.  Large 
companies  in  particular  turn  to  supervision  and 
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tcclinicai  specialists  tor  the  sokitioii  to  problems, 
ignoring  the  know-how  and  creative  ability  ot  the 
worker.  Since  the  worker  is  .seldom  asked  for  bis 
opinion  or  for  his  solution  to  a  problem,  is  it  anv 
wonder  that  he  feels  that  no  one  cares?  And  it  no  one 
cares,  how  can  he  take  pride  in  his  crattsmansh ip? 

Another  contributing  factor  to  the  lack  of  personal 
pride  in  craftsmanship  is  that  sometimes  there  is  no 
pride  to  be  had.  This  can  be  caused  bv  boredom, 
where  the  work  is  coo  simple,  too  repetitive,  or  the 
worker  has  been  doing  the  same  thing  for  so  long  that 
he  has  lost  interest.  This  problem  can  be  solved  by 
reassigning  the  individual  to  a  more  suitable  job. 

An  absence  ot  personal  pride  can  also  occur  when 
the  individual  does  not  possess  the  necessary  skills  to 
perform  the  job  satisfactorily.  This  can  usually  be 
corrected  bv  on-the-job  training.  Most  people  who  are 
not  adequatclv  trained  tor  a  job  will  admit  to  their 
problem  if  they  know  someone  is  trying  to  help 
them.  Quality  control  trend  charts  are  a  very  effective 
tool  tor  identifying  the  person  with  this  problem. 
Repetitive  errors  of  the  same  nature  more  than  likely 
mean  that  some  form  of  training  is  required. 

Fortunately,  industry  has  begun  to  recogni/.e  the 
problem  of  a  lack  of  pride  in  personal  craftsmanship 
and  some  noteworthy  action  has  been  taken.  The 
Quality  Circle  program  is  a  good  e.\ample  of  how 
pride  in  craftsmanship  can  be  stimulated  and  put  to 
use.  A  Quality  Circle  is  a  small  group  of  people  who 
do  similar  work  and  meet  regularly  to  identify, 
analyze,  and  develop  solutions  to  work-related 
problems.  Companies  using  this  program  are  reporting 
impressive  results  in  improved  quality  of  workman¬ 
ship.  cost  reduction,  and  reduced  lost  time.  Because 
of  its  employee-oriented  nature,  the  program  provides 


the  worker  the  opportunity  tor  personal  development 
and  recognition.  The  result  is  a  significant  improve 
ment  in  morale  and  craftsmanship. 

Direct  communication  from  management  to  the 
worker  is  a  simple  but  verv  ettective  means  ot 
improving  the  wtirker's  image,  his  morale,  and  his 
pride  in  craftsmanship.  A  periodic  meeting  that 
informs  the  worker  where  he  tits  into  the 
organization,  what  is  the  line  ot  command,  the  st.itus 
of  existing  programs,  the  future  business  outlook,  .nui 
immediate  program  goals  makes  the  worker  teel  that 
he  is  a  recognized  and  intormed  member  ot  a  team. 
This  easy-to-implement  man.igement  tool  never  tails 
to  stimulate  pride  in  crattsmansh  ip. 

In  summary,  I  believe  that  pride  in  craftsmanship 
does  still  exist,  but  it  needs  to  be  revitalized.  This  can 
be  achieved  bv  the  following  course  of  action: 

•  Don't  tic  the  craftsman's  h.mds  with  excessive 
direction  and  control. 

•  Polish  the  craftsman's  im.ige.  Give  credit  tor  a 
job  well-done,  and  stilicit  his  opinion  on 
production  problems. 

•  Keep  the  craftsman  intormed.  M.ike  him  feel 
that  he  is  a  member  ot  a  team  and  tliat  his  role  is 
important. 

•  Make  the  craftsman  aware  ot  quality  control 
trends  and  problem  areas. 

•  Be  on  the  lookout  for  substandard  performance 

and  have  training  courses  reach  for 

implementation. 


CRAFTSMANSHIP  WORKSHOP: 

Subgroup  Discussions 
DISCUSSION  PANEL  CO-CHAIRMEN: 

N.  S.  Wilson 
J.  F.  Otto 
D.  Verrastro 
R.  Egelston 
F.  I.  Given 
ISSUES: 

•  Age  of  Workforce 

•  Training 

•  Qualification 

•  Recognition 
RECOMMENDATIONS: 

Age  of  the  workforce  and  training  were 
treated  as  one  subject.  It  was  recom¬ 
mended  that  industry  implement  the  follow¬ 
ing  type  plan  for  developing  mechanics, 
technicians  and  technical  staff  so  that 
retiring  workers  can  be  replaced  with 
minimum  impact: 

•  Coordinate  with  local  school 
officials,  minority  leaders, 
unemployment  offices  to  identify 
potential  trainees. 


Qualification  was  considered  to  be  a 
minor  problem  that  can  be  controlled  by 
two  actions: 

•  Update  job  descriptions 

•  Enforce  demonstration  of  skill 
proficiency  during  probationary 
period 

Recognition  of  individual  performance 
was  unanimously  agreed  to  as  an  effective 
tool  for  motivating  the  work  force. 

Formal  programs  should  be  developed  and 
implemented  by  all  contractors.  Some 
successful  techniques  were  identified: 

•  Astronaut  visits 

•  Management  talks 

•  Trips  to  launch  sites 

•  Dinners 

•  Pin  presentations  (such  as 
"Snoopy"  type) 

0  Certificates  of  achievement. 


0  Provide  training  for  required 
skills  utilizing  classroom  and 
on-the-job  programs. 

0  Direct  the  training  along  the 
line  of  career  development. 
Encourage  schools  to  provide  such 
training  as  a  regular  part  of 
their  cirriculum. 

0  Investigate  potential  of  training 
"pink  collar  workers"  such  as 
secretaries,  to  become  planners, 
schedulers,  cost  analysts,  etc. 
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QUALITY  VS  SCHEDULE  TRADEOFF 
Art  Huschle 

Product  Assurance  Manager 
Honeywell,  Inc.,  Defense 
Electronics  Division 

We  have  all  been  faced  with  making 
tradeoffs  between  quality  and  schedule 
during  our  careers.  Many  of  us  can  even 
point  out  some  times  when  quality  was 
traded  off  to  meet  schedule  and  no  major 
repercussions  occurred.  But  I  am  willing 
to  bet  that  the  majority  of  the  time  when 
you  have  traded  quality  for  schedule,  it 
has  cost  -  in  dollars,  in  performance,  in 
customer  satisfaction  and/or  in  schedule. 

One  of  the  major  reasons  for  having  sched¬ 
ule  problems  is  due  to  lack  of  or  poor 
planning.  The  best  way  to  avoid  quality 
vs  schedule  tradeoffs  is  to  improve  on 
quality  planning. 

Today,  I  want  to  tell  you  about  our  tech¬ 
nique  that  we  at  Honeywell,  Training  and 
Control  Systems  Center  have  been  utilizing 
which  has  avoided  many  quality  vs  schedule 
tradeoffs.  The  technique  is  called 
"Integrated  Test  Planning." 

INTEGRATED  TEST  PLANNING 

General :  Conceptually,  Integrated  Test 
Planning  is  a  comprehensive  process  for 
mating  the  planning,  testing  and  correlat¬ 
ing  of  the  resulting  test  data.  A  prop¬ 
erly  integrated  test  program  defines  the 
constraints,  objectives,  responsibilities 
and  techniques  for  planning,  implementing, 
monitoring  and  controlling  the  test  and 
evaluation  of  the  product.  It  provides 
for  the  orderly  dissemination  and  utili¬ 
zation  of  test  data”' 

The  Integrated  Test  Planning  process, 
essentially,  has  three  phases: 

1.  Planning 

2.  Implementation 

3.  Reporting  and  Feedback. 

(l)Ref :  "Integrated  Product  Testing  and 
Evaluation:  A  systems  approach 
to  Improve  Reliability  and  Qual¬ 
ity"  by  H.  Gilmore  and  H. 
Schwartz,  1969. 


Objectives  and  Requirements :  To  ensure 
a  timely,  efficient  and  cost  effective 
test  program.  The  Integrated  Test  Plan 
provides  the  roadmap  to  personnel  for 
fulfilling  these  overall  objectives. 

The  ITP  is  a  formal  document  which 
establishes: 

•  what  must  be  done 

•  who  must  do  it 

•  when. 

Gilmore  and  Schwartz^^^  identified  the 
following  objectives  which  are  basic  to 
any  well -planned  test  program: 

(a)  Detect  engineering  deficiencies 
as  early  as  possible  to  permit 
suitable  corrective  action. 

(b)  Test  in  the  most  realistic 
operation  environment  available. 

(c)  Eliminate  redundancy  of  test 
effort. 

(d)  Provide  as  much  flexibility  as 
possible  to  allow  for  inevita¬ 
ble  changes  which  cannot  be 
predicted,  for  example,  fail¬ 
ure,  with  subsequent  retest 
requirements. 

(e)  Ensure  that  critical  test  items 
will  not  be  subjected  to  exces¬ 
sive  or  unnecessary  operation. 

(f)  Promote  orderly  progression 
from  laboratory  test  through 
operational  use  on  schedule. 

(g)  Optimization  of  test  specimen 
cost  through  maximum  use  of  all 
test  specimens. 

(h)  Continuous  control  of  test 
specimen  configuration. 

(i)  Provide  a  basis  for  the  corre¬ 
lation  and  utilization  of  test 
data  from  all  sources. 


(j)  Pennit  orderly  and  controlled 
implementation  of  the  test  pro¬ 
gram  with  logical  timing, 
sequencing,  and  interrelation 
of  the  several  types  of  tests. 

(k)  Correlation  of  discrepancies 
and  failure  experience  from 
test  to  test,  but  primarily 
from  development  through  final 
acceptance  testipg. 

(l)  Orderly  development  and  verifi¬ 
cation  of  requiremerits  and 
procedures  in  parallel  with 
the  hardware  development. 

Honeywell's  Approach:  Honeywell  approach 
is  not  that  different  or  unique  from  the 
fundamentally  sound  approach  just  pre¬ 
sented.  What  is  different  or  unique  per¬ 
haps  is  its  objectives  and  requirements. 
These  objectives  include: 

1.  Early  emphasis  on  test  planning 
via  a  fonnal  program  Integrated 
Test  Plan  (ITP)  and  detail  Test 
Plan  Summary  Sheets  (TPSS). 

2.  Establish  an  organization  element 
with  singular  responsibility  for 
T&E. 

3.  Eliminate  redundant  testing. 

4.  Identification  and  early  imple¬ 
mentation  of  engineering  investi¬ 
gative  tests. 

5.  Establish  a  corss-reference  of 
test  requirements  that  tracks 
from  the  highest  to  the  lowest 
hardware  level. 

6.  Emphasize  early  identification 
of  problems  and  their  timely 
corrective  action. 

7.  Consider  ITP  a  working  document; 
periodic  updating. 

These  objectives  provide  the  major  theme 
behind  a  successful  Test  Program  and  that 
theme  is: 

"To  deliver  equipment  at  minimum 

cost,  on  schedule,  and  with  minimal 

technical  risk" 


INTEGRATED  TEST  PLAN 

Honeywell  considers  the  following  elements 
as  basic  to  formation  of  a  typical  inte¬ 
grated  test  plan.  It  should  be  noted  that 
the  type  of  program  (developmental  or 
production)  will  effect  the  emphasis  and 
detail  contents  of  each  section: 

Typical  ITP  contents: 

Objective  Test  Support 

ITP  Update  Test  Documentation 

Organization  Schedules 

Test  Program 

OBJECTIVE: 

•  Identify  major  test  program 
objectives. 

•  Denote  type  of  program: 

Production : 

Emphasize  detail  scheduling  and 
availability  of  test  equipment; 

Concerned  with  Manufacturing 
Screening,  Factory  Acceptance 
Tests,  Production  Assessment 
tests;  and  STE  Design  Evaluation/ 
Operational  Proofing  Tests. 

Developmental : 

Emphasis  on  detail  planning  from 
lowest  hardware  level  through  to 
highest; 

Concerned  with  Engineering  Invest¬ 
igative,  EMC,  Environmental 
Extremes,  Reliability  and  other 
Subsystem  (or  System)  level  Engi¬ 
neering  Evaluation  tests  prior  to 
Operational  Testing. 

ITP  UPDATES: 

•  Specify  periodic  updates, 
typically: 

•  Quarterly  during  a  production 
program  and  yearly  during  a 
development  program. 

•  Provide  updates  in  an  Integrated 
Test  Status  Report: 
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•  In  development  programs  can 

be  provided  as  part  of  monthly 
technical  status  report. 

•  The  Status  Report  should 
include  the  following  as  a 
minimum: 

Brief  discussion  of  signifi¬ 
cant  problems. 

Updates  of  test  schedules. 

List  of  tests  completed. 

List  of  failures  having 
occurred,  including  summary 
results  of  analyses  performed 
and  corrective  action. 

Changes  to  ITP. 

ORGANIZATION: 

•  Assignment  of  a  single  individual 
in  the  program  organization  with 
responsibility  for  the  test  pro¬ 
gram.  He  should  report  directly 
to  the  Program  Manager  and  func¬ 
tion  to: 

•  Implement  test  program  plan 
and  schedules; 

•  Coordinate  availability  of 
test  resources,  including  usage 
of  facility  support  groups; 

t  Provide  direction  in  solving 
test  problems,  responsible  for 
arriving  at  corrective  action; 

•  Advise  program  management  of 
progress,  schedule  constraints 
and  problems. 

TEST  PROGRAM: 

•  Clearly  identify  all  tests,  relat¬ 
ing  to  the  various  hardware 
levels : 

•  LRU,  Major  Component,  Subsys¬ 
tem  and/or  System; 

•  Refer  to  Figure  1. 


•  Emphasis  on  setting  up  a 
progressive  sequence  of  testing 
at  increasing  levels  of  assem¬ 
bly  and  environmental 
complexity. 

•  Preparation  of  detail  Test  Plan¬ 
ning  Summary  Sheets  (TPSS)  for 
all  test  items; 

•  The  purpose  of  the  TPSS  is  to 
document  the  test  requirements, 
test  objective,  test  equipment, 
test  description,  pass/fail 
criteria,  and  test  schedule 

(as  appl icable) . 

•  Two  examples  are  shown;  both 
are  applicable  to  development 
programs: 

Example  #1:  TPSS  has  a  broader 
scope  of  usage  and  closely 
follows  the  elements  listed 
above. 

Example  #2:  is  performance 
oriented  and  in  this  respect 
is  very  detailed,  specifying 
the  planned  methods  for  veri¬ 
fying  the  requirements. 

•  Clearly  shows  all  parties  the 
intended  purpose  and  interpret¬ 
ation  of  the  test  requirements. 
Provides  some  planning  inform¬ 
ation  (i.e.,  equipment, 
schedule) . 

•  Types  of  Tests: 

Development  Test  Program  -  typi¬ 
cally  includes  the  following: 

•  Development  or  Engineering 
Investigative  (subsystem, 
major  component,  LRU  -  bread¬ 
boards  and  prototypes) ; 

•  Manufacturing  Tests  (system, 
major  component,  LRU): 

Receiving  Tests 

Parts  Screening 

Product  Acceptance,  including 
software 


Manufacturing  Screening  Tests. 


TEST  DOCUMENTATION 


t  Engineering  Evaluation  or 

Design  Verification  (subsystem, 
and  system) : 

Environmental  Qualification 

EMC,  includes  EMI,  Magnetic, 
EMCON,  EMP,  etc. 

R/M  Demo's 

Performance  Evaluation 

•  Operational  Testing  (System) 

Production  Program  -  typically 
includes: 

•  Manufacturing  Tests  - 
Parts  Rescreen 

Burn  In 

Random  Vibration 
Thermal  Cycle 
Acceptance 

t  First  Article  Inspection/ 
Production  Qualification 
(system) 

•  Production  Assessment  Tests 
(system) 

•  Design  Evaluation  Tests  and 
Operational  Proofing  of  STE. 

ff  Clearly  establish  test  responsi¬ 
bilities  (Refer  to  attached 
example) . 

•  Specify  the  data  collection, 
reduction  and  analysis  require¬ 
ments.  Where  applicable  prepare 
a  plan. 

TEST  SUPPORT 

•  Identify  Test  Facilities  and 
Equipment,  including  GFE. 

•  Personnel 

•  Logistics  in  support  of  System 
Level  field  testing. 


•  List  all  test  procedures  and 
reports  that  will  be  required. 

•  Procedures  should  provide 
detailed  step-by-step  procedures 
for  performing  test;  include  data 
requirements. 

•  Consider  usage  of  Flash  Test 
Reports : 

•  Vehicle  used  to  inform  the 
internal  engineering  organiza¬ 
tion  immediately  on  test 
results ; 

•  Format  to  address  test  pur¬ 
pose,  parameters  measured, 
results  and  impact  statements 
as  to  effect  on  the  collected 
data ; 

•  Compile  within  one  day  of 
test; 

•  About  one  page  in  length. 
SCHEDULES 

•  Coordinate  with  Master  Program 
Schedule 

•  Must  schedule  in  detail  all  tests 
including  submittals  of  test  pro¬ 
cedures  and  reports 

•  Via  the  integrated  test  status, 
report  and/or  ITP  update,  process 
schedules  should  be  periodically 
updated.  Provide  status  and 
important  planning  information. 

Integrated  Test  Planning  will  not  elim¬ 
inate  the  need  to  perform  quality  vs 
schedule  tradeoffs.  If  properly  done, 

ITP  will  reduce  the  number  of  tradeoffs 
during  a  program  and  will  also  prove  to 
be  an  invaluable  tool  to  assess  the  impact 
of  the  tradeoffs. 


EXAMPLE  #  1 


TEST  PLAN  SUftlARY  SHEET 


TEST  NO:  FOOOl-C 

TEST  TITLE  Reliability  Demonstration  Test  (RDT) 

TEST  ITEM  EDM  2  and  EDM  3 

PART  NO.  _ _ _ _ 

TEST  OBJECTIVE:  To  demonstrate  by  test  that  the  system  satisfies  the 

MTBF  and  MTTF  requirements  of  the  System  Specification. 

TEST  REQUIREMENT  DOCUMENT: 

System  Specification;  R/M  Program  Plan;  RDT  Test 

Procedure  (to  be  developed) 

TEST  DESCRIPTION: 

The  primary  system  MTBF  and  the  MTTF  requirements  will  be 
demonstrated  in  accordance  with  Test  Plan  HI  of  MIL-STD-781 
(sequential  test,  x=  ^  =  10;',  discrimination  ratio  2:1);  the 
PM/FL  MTBF  and  Yaw  stabilization  MTBF  will  be  demonstrated  in 
accordance  with  Test  Plan  IV  of  MlL-STD-781  (sequential  test, 
jfi^=  p  -  20'“.,  discrimination  ratio  2:1). 

TEST  INSTRUMENTATION  REQUIREMENTS: 

VTVM,  Multimeter,  Oscilloscope,  System  Special  Test  Equipment, 
Tools  for  fault  repair,  spares  and  consummables . 


PASS/FAIL  CRITERIA: 

4. 2. 8. 3  of  MIL-STO-781  for  the  Primary  System  MTBF  and  the  MTTF; 

4. 2. 8. 4  OF  MlL-STO-781  for  the  PM/FL  MTBF  and  the  Yaw  Stabilization 
MTBF. 


SCHEDULE: 


START _ April  1974  COMPLETE^  _  iuTL®. 
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EXAMPLE  H  2 
SEAFIRE  TEST  PLAN  SUMMARY  SHEET 


ms  NO.  A-oooi 


PART  NUMBER 


APPLICABLE  SPECIFICATION 


TEST  LOCATION  T* •*CSC 


100% 


RtqwirMi  MvnwfKtvrlnf  TMt* 


Critvfi* 


PERFORMANCE 

1.  Ilnterface  definition: 

.  interfaces 

environmental  conditions 
input/output  signal  characteristics 
shielding 

self-test  signals  (BIT) 

primary  power  interface  (per  4.2.1  3.3.2) 
director  angular  coverage 
man/machine  interface 
tilt  compensation  (and  sub  panels) 
azimuth  and  elevation  aligiment 
.  power  (per  4.2.1  3.3.1) 

OPERATIONAL  STATES: 

OFF  state 
STANDBY  state 
.  OPERATE  state 

-  ready 

-  designation/slave 


P  -  Periodic  Verification 


3.1.5 

3. 1.5. 1.1 
3. 1.5. 1.1. 4 

3. 1.5. 1.2.1 

3. 1.5. 1.2. 2 

3. 1.5. 1.2. 3 

3. 1.5. 1.3.1 
3. 1.5.1. 4. 5i 
3. 1.5. 1.8 

3. 1.5. 2. 2 

3. 1.5. 2. 3 

3. 1.5. 3 

3.2.1. 1 

3. 2. 1.1.1 

3. 2. 1.1. 2 

3. 2. 1.1. 3 

3.2.1. 1.3.1 

3. 2. 1.1. 3. 2 


C  -  Continuous  Verification 


*  =  Verify  according  to  room  ambient  (final  acceptance)  test  method. 

•*  =  Verify  in  accordance  with  4. 2. 1.3.1  of  System  Specification,  unless  otherwise  specified. 


TABLE  1.  TEST  PFSPONSIBILITV  MATPI)! 


— — Jest  Element 
Test  Phase  — 

Test 

Plan 

Site 

Prep. 

_ 

1  Test  .Equip.  .  Orq. 

l&C;Director  Operation  ftaint. 

L_  _  i  _  .  _ L 

Data 

Collection 

Data 

Analysi s ; 

Test 

Reponts 

Equipmeri 

Teardewn 

All  MA  Tests 

H 

H 

H 

H 

H 

H 

i  ! 

H  1  H  ' 

1 

H 

H 

Subsystem  Environ. 

H 

li 

H 

H 

H 

H 

H 

H 

H 

H 

EMI  (MIL-STD-46i) 

H 

H 

H 

H 

H 

H 

H  ! 

H 

H 

EMP 

H 

H 

H 

H 

H 

H 

li  : 

H 

H 

Reliability 

Devel opment 

H 

H 

H 

H 

H 

H 

H 

h 

H 

H 

Ma  inta  inabi  I  i  ty 
Verification 

H 

H 

H 

M 

H 

H 

H  i 

H 

Rel iabi 1 i ty 
Demonstration 

H 

H 

H 

H 

H 

H 

H 

1 

G  ! 

G 

Maintainabil ity 
Demonstrat ion 

G 

G 

n/< 

G 

G 

6 

G 

G 

G 

n/a 

EMCON/EMC 

G 

G 

H 

G 

G 

H 

G 

1 

'  ! 

G 

H 

ECCM 

G 

H 

G 

G 

H 

G 

G  1 

G 

H 

Performance  Eval 
Test; 

Phase  1 

H 

G 

H 

H 

H 

H 

H 

i 

H  1 

H 

H 

Phase  11 

G 

n/a 

n/i 

G 

H 

G 

G  ! 

G 

n/a 

Phase  III 

G 

n/a 

n/< 

G 

G 

G 

G 

G  i 

G 

H 

Operational  Test 

G 

G 

H 

G 

G 

G 

G 

2  1 

G 

H/G 

Definition  of  Terms;  H  =  Honeywell;  G  =  Government;  n/a  =  Not  Applicable. 
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TRAINING  FOR  CRAFTSMANSHIP 

R.  W,  Egelston,  P.E. 

Electronics  &  Optics  Division 
The  Aerospace  Corporation 

What  is  craftsmanship.  Quoting  from 
Webster's  Unabridged  Dictionary  "it 
is  the  result  we  see  when  a  craftsman 
does  work  of  consistently  high  quality". 
Substituting  manufacturing  operator 
for  craftsman  and  high'  reliability  for 
high  quality,  we  have  a  definition  that 
suits  our  purpose. 

In  this  paper  I  will  first  review  the 
industry  practice  for  developing  oper¬ 
ators,  make  some  suggestions  for  im¬ 
proving  the  level  of  craftsmanship,  and 
conclude  by  considering  what  has  to  be 
provided  to  support  the  operator  in  pro¬ 
ducing  high  rel  hardware. 

Now  before  we  proceed  with  Mission 
Assurance  1980,  you  might  like  to  look 
at  Mission  Assurance  1850! 

VIEW  GRAPH  #  1 

So  much  for  nostalgia. 

When  I  was  scheduled  to  talk  on  the  sub¬ 
ject  of  Training  for  Craftsmanship,  I 
recalled  how  often  "  operator  error" 
was  reported  as  the  cause  for  discrep¬ 
ancy  on  MRB  and  waiver/deviation 
actions,  and  the  corrective  action  was, 
"cautioned  the  operator",  "retrained 
the  operator",  "disciplined  the  oper¬ 
ator",  etc. 

In  a  majority  of  these  cases,  it  was  my 
judgement  that  the  real  culprit  was 


insufficient  training  of  the  operator, 
possibly  compounded  by  too  lax  super¬ 
vision. 

So  it  seems  to  me,  that  there  must  be 
a  better  way  to  train  the  people  who 
perform  manufacturing  and  assembly 
operations. 

Present  Industry  Practice 

Applicants  with  minimum  qualifications 
are  given  from  one  to  six  weeks  of 
classroom  training  in  an  assembly 
school  before  continuing  on-the-job 
training  on  the  production  line.  The 
more  apt  trainees  are  sent  to  programs 
that  require  higher  levels  of  compet¬ 
ence,  etc.  ,  so  that  practically  all  en¬ 
trants  are  eventually  employed.  Thus 
there  is  no  psychological  incentive  to 
strive  for  perfection. 

In  theory,  the  line  supervisor  takes 
over  the  trainee  and  structures  his  or 
her  work  assignments  to  fit  the  capa¬ 
bilities  of  the  individual,  gradually  in¬ 
creasing  task  difficulty  and  complexity 
until  the  trainee  has  developed  into  a 
competent  operator. 

Since  it  requires  a  minimum  of  six 
months  to  thoroughly  train  an  operator, 
it  is  obvious  that  most  of  the  operator's 
skills  are  developed  practicing  on  pro¬ 
duction  hardware! 

In  practice,  the  supervisor  may  be 
spread  too  thinly  to  give  the  individual 
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attention  necessary  to  make  the  pro¬ 
gram  work  as  desired.  Thus,  as  I  re¬ 
marked  before;  "operator  error"  re  - 
suits. 

My  remarks  are  not  intended  to  indict, 
but  rather  constructively  critize  these 
training  procedures  and  hopefully  bring 
about  improvements  in  "Training  for 
Craftsmanship.  " 

Proposed  Hi  Rel  Operator  Training 
My  proposal  is  to  establish  a  manu- 
fac  turing  operations  school,  where  all 
facets  of  hardware  manufacturing  and 
assembly  are  taught  prior  to  the  oper¬ 
ator's  taking  his/her  place  on  the  pro¬ 
duction  line.  This  will  be  a  one  level 
of  competence  school,  program  orient¬ 
ed  to  produce  nothing  but  hi  rel  hard¬ 
ware. 

The  candidates  for  this  training  would 
be  selected  by  industrial,  psychological 
and  physical  testing  for  the  character¬ 
istics  necessary  to  develop  into  a  hi 
rel  operator,  such  as; 

•  Natural  Aptitude 

•  Manual  Dexterity 

•  Attitude  Toward  the  Work 

•  Emotional  Stability  and  physical 
capacity  to  perform  at  a  high 
level  consistently 

Not  all  people  who  want  to  learn  manu¬ 
facturing  operations  are  suitable  for  hi 
rel  work. 

The  instructors  must  be  outstanding  in 


their  ablility  to  instill  the  hi  rel  attitude 
in  their  trainees. 

From  the  first  day  of  training  the  future 
operators  must  be  indoctrinated  in  the 
importance  of  their  contribution  to  the 
end  product. 

Somewhere  in  their  training  they  should 
learn  what  the  finished  product  is,  its 
importance  to  national  security,  and 
where  their  particular  operations  fit  in 
the  overall  picture. 

Examples  of  essentially  perfect  hard¬ 
ware  should  be  displayed  and  the  trainee 
should  have  enough  "hands  on"  practice 
to  produce  hardware  that  approximates 
the  samples. 

This  should  not  be  a  structured  "X" 
number  of  hours  training  program,  but 
rather  flexible  enough  to  allow  attain¬ 
ment  of  near  perfection  before  the  oper¬ 
ator  works  on  production  hardware. 

The  passing  grade  for  the  hi  rel  oper¬ 
ator  should  be  in  the  high  90' s.  70%  is 
not  good  enough  for  a  hi  rel  program. 

But,  training  for  craftsmanship  doesn't 
stop  on  graduation  day.  The  line  super¬ 
visor,  takes  over  the  responsibility  of 
keeping  the  high  rel  attitude  alive. 

VIEW  GRAPH  #2 

Now  let's  itemize  some  of  the  support 
items  needed  to  assure  manufacturing 
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the  quality  of  product  necessary  to 
assure  Mission  Success. 

TRAINIMG  FOR  CRAFTSMANSHIP 

o  DESIGN  FOR  EASE  OF  FABRI¬ 
CATION 

o  APPROPRIATE  WORK  STATIONS 
o  COxMFORTABEE  ATMOSPHERE 
o  DETAILED  ASSEMBLY  INSTRUC¬ 
TIONS 

o  ASSEMBLY  AIDS 
o  COMMITTED  MANAGEMENT 

o  CONTINUING  INCENTIVES 

DESIGN  FOR  EASE  OF  FABRICATION 

The  manufacturability  of  the  hardware 
should  be  influenced  as  early  as  poss¬ 
ible  in  the  design  stage. 

If  the  design  is  such  that  parts  place¬ 
ment  is  difficult  or  attachment  (solder, 
weld,  etc.  )  is  troublesome,  reliability 
may  degrade. 

APPROPRIATE  WORK  STATIONS 

The  work  station  should  fit  the  oper¬ 
ation  being  performed,  being  outfitted 
with  necessary  tools,  materials,  etc. 

COMFORTABLE  ATMOSPHERE 

Lighting,  air  conditioning;  clean  room 
atmosphere  if  required;  all  that  is  need¬ 
ed  to  support  the  high  rel  attitude. 


DETAILED  ASSEMBLY  INSTRU  CTIONS 

This  goes  with  "appropriate  work 
stations"  but  is  so  important  that  it 
m.ust  be  emphasized.  The  details  of 
assembly  must  be  spelled  out.  What 
tools,  or  materials  are  to  be  used? 

How  they  are  to  be  used?  The  assem¬ 
bly  planning  function  is  the  bridge  be¬ 
tween  the  operator's  skill  and  the 
finished  high  rel  assembly.  "Clean  as 
required"  doesn't  cut  it! 

ASSEMBLY  AIDS 

Another  item  that  goes  v/ith  "appro¬ 
priate  work  stations".  Any  jig,  fixture, 
picture,  etc.,  that  assures  proper  as¬ 
sembly  will  enhance  our  quest  for  craft- 
manship. 

COMMITTED  MANAGEMENT 

The  high  rel  attitude  must  permeate 
all  levels  of  management,  but  the  role 
of  the  line  supervisor  is  most  critical. 
He  is  the  one  who  establishes  a  coach/ 
player  relationship  with  the  operator, 
and  he  must  not  be  loaded  with  so  much 
paperwork  and/or  so  many  operators 
that  he  has  no  time  to  supervise. 

CONTINUING  INCENTIVE 

The  continuing  incentive  for  maintain¬ 
ing  excellence  is  recognition  of  the 
operator's  individual  contribution. 
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SUMMARY 


The  elements  of  Training  for  Crafts¬ 
manship  are: 

.  Select  trainable  people.  Thorough¬ 
ly  develop  their  skills. 

.  Indoctrinate  them  with  the  high 
rel  attitude. 

.  Provide  the  support  necessary  to 
produce  high  rel  hardware. 

.  Provide  continuing  education 

and  reinforcement  of  the  high  rel 
attitude. 

.  Recognize  the  individual's  achieve¬ 
ment  and  contributions. 

.  Motivate  the  operator  to  continue 
performing  at  the  highest  poss¬ 
ible  level. 


In  conclusion,  to  paraphrase  a  favorite 
saying  in  our  industry,  "Make  it  right 
the  first  time  for  lower  costs  and  higher 
reliability". 


TRAINING  FOR  CRAFTSMANSHIP 

DESIGN  FOR  EASE  OF  FABRICATION 
APPROPRIATE  WORK  STATIONS 
COMFORTABLE  ATMOSPHERE 
DETAILED  ASSEMBLY  INSTRUCTIONS 
ASSEMBLY  AIDS 
COMMITTED  MANAGEMENT 
CONTINUING  INCENTIVES 
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DEVELOPMENT  OF  ELECTRONIC 
WORKMANSHIP  INSPECTION  STANDARDS 
FOR  SPACE  APPLICATION 
JACK  D,  REBERRY 
STAFF  QA  SPECIALIST 
USAF  -  SPACE  DIVISION 

The  first  thing  that  comes  to  mind 
when  one  hears  that  another  military 
specification  is  being  written  is  why? 
There  are  specifications  and  standards 
written  for  just  about  anything  that  can 
be  imagined  -  including  space  application. 
However,  as  you  may  or  may  not  know,  the 
first  Mission  Assurance  Conference  un¬ 
covered  the  fact  that  though  many  specifi¬ 
cations  and  standards  having  to  do  with 
workmanship  have  been  published  by  the 
Government  only  a  few  actually  had  cri¬ 
teria  that  Illustrated  and  described  what 
was  acceptable  or  unacceptable. 

Therefore,  on  the  fifth  of  March,  1979 
the  Air  Force  Liaison  Panel  of  the  Aero¬ 
space  Industries  Association  (AIA)  and  the 
National  Security  Industrial  Association 
(NSIA)  met  with  Space  Division  to  discuss 
this  mutual  problem.  One  of  the  chief 
concerns  of  Space  Division's  Commander, 

Lt  Gen  Richard  Henry,  was  and  is  getting 
good  craftsmanship  (workmanship)  on  space¬ 
craft. 

At  that  meeting  AIA  and  NSIA  agreed  to 
take  on  the  task  of  developing  a  set  of 
good  workmanship  standards  that  could  be 
used  to  bridge  the  requirements  between 
specifications  and  assembly  drawings  or 
Instructions.  Marty  Adams,  Rockwell  In¬ 
ternational  and  George  McGee,  Martin 
Marietta  agreed  to  cochair  a  working  com¬ 
mittee  and  AIA,  and  NSIA  agreed  to  fund 
the  project.  It  was  assigned  project  No. 
79.3. 

The  objective  of  the  project  was  to 
dev.elop  and  provide  a  coordinated  set  of 
workmanship  standards  that  described 
Space  Division's  criteria  for  acceptance 
of  high  reliability  electronic  products. 
The  scope  of  the  project  was  to  be  limit¬ 
ed  to  pre-installation,  installation  and 


post  installation  of  parts  used  on  space¬ 
crafts  and  boosters,  and  to  form  the 
basis  fcr  a  military  specification. 

AIA/NSIA  described  the  project  as  fol¬ 
lows: 

"A  large  number  of  documents 
dealing  with  the  subject  of 
workmanship  standards  have 
been  issued  by  various  Govern¬ 
ment  agencies.  Many  are  re¬ 
dundant  or  conflicting;  many 
attempt  to  describe  "how  to" 
do  a  Job  rather  than  to  des¬ 
cribe  criteria;  some  techno¬ 
logies  have  not  been  treated; 
almost  every  Aerospace  company 
has  their  own  set  of  standards 
which  may  or  may  not  be  accep¬ 
ted  by  the  Government  as  ade¬ 
quate.  This  project  will  at¬ 
tempt  to  provide  a  single  set 
of  standards  to  be  used  by 
SAMSO  for  future  procurements." 

"Maximum  use  will  be  made  of 
line  drawings  and  artists' 
renditions  of  specific  stand¬ 
ards  with  appropriate  simple 
text  to  describe  acceptance 
and  rejection  criteria." 

The  task  description  went  further  by  sta¬ 
ting  that  the  goal  would  be  to  develop  a 
workmanship  standard  that  could  be  used 
by  military  and  industry  and  would  elim¬ 
inate  uncertainties  that  now  exist  in 
Government  standards.  Industry  believes 
present  Government  standards  are  incom¬ 
plete  and  not  current  with  modern  assem¬ 
bly  and  packaging  methods. 

A  committee  was  then  formed  to  outline 
schedules  to  accomplish  the  tasks,  find 
knowledgeable  workers  who  were  willing  to 
accept  the  responsibility  of  developing 
the  standards,  and  schedule  other  necessary 
events. 

The  committee  defined  that  the  standards 
must  avoid  describing  "how"  to  accomplish 
any  specific  task.  To  tell  manufacturers 
and  their  employees  how  to  do  a  job,  we 
believe,  takes  away  the  incentive  to  im¬ 
prove  on  a  manufacturing  process;  that  is, 
to  do  it  better,  faster,  or  less  expensive. 


Instead,  these  standards  are  to  only 
direct  the  inspector,  assetnblier  or  opera¬ 
tor  in  deciding  good  from  bad,  acceptable 
from  rejectable.  They  are  to  have,  to 
the  greatest  extent  possible,  clear  vis¬ 
ual  drawings  or  pictures,  or  both,  and  as 
little  narrative  as  possible  to  describe 
the  acceptable  and  unacceptable  criteria. 
When  known,  good  reference  notes  may  also 
be  Included.  The  same  is  true  for  appli¬ 
cation  notes.  When  possible,  all  stan¬ 
dards  should  contain  four  examples-two 
acceptable-two  unacceptable-arranged  in 
decreasing  order  from  near  perfect  to 
very  bad.  They  should  be  titled  PREFER¬ 
RED,  ACCEPTABLE  and  UNACCEPTABLE.  There 
should  always  be  one  preferred  and  at 
least  one  acceptable  and  one  unacceptable. 
Either  of  the  following  combinations 
should  be  used. 


Preferred 

Acceptable 

Acceptable 

Unacceptable 


Preferred 
or  Acceptable 
Unacceptable 
Unacceptable 


All  dimensions  are  also  to  be  in 
metric. 


On  May  1,  1979  a  meeting  was  held  at 
Rockwell  International,  Seal  Beach,  CA. 
Present  were  George  McGee,  representing 
AIA;  Marty  Adams,  representing  NSIA  and 
myself  representing  Space  Division.  The 
purpose  of  the  meeting  was  to  specifically 
define  the  areas  of  workmanship  that  would 
be  covered  by  the  project,  select  a  com¬ 
mittee  to  draft  the  standards  and  establish 
the  considerations  (characteristics)  for 
each  of  the  selected  areas.  Eight  areas 
(manufacturing  processes)  were  identified, 
they  were: 

1.  Micro  circuit  installation. 

2.  Transistor  installation. 

3.  Mounting  of  small  chokes  and  trans¬ 
formers. 

4.  RF  circuits  (special  consideration) 
(subsequently  canceled). 

5.  Pretinning  of  component  leads  and 
removal  of  gold  plating. 

6.  Deflnitlzed  acceptance  criteria  and 
visual  standards  for  printed  cir¬ 
cuit  board  acceptance  after  compon¬ 
ent  installation. 

7.  Jumper  wire  installation  on  PCB's 

8.  Conformal  coating. 


Besides  the  two  cochairmen,  Marty 
Adams  and  George  McGee,  the  original 
committee  was  made  up  of  the  following 
personnel: 


G.  F.  Thomas 
John  Churchwell 
Charles  C .  Cook 
Ben  E.  Graves 
Herman  L.  Wuerffel 
Jack  Janoyan 
Carl  J.  Thaler 

Tver  Wahl 

Jack  Reberry 

George  Adams 

Bill  Coey 
Tom  Mayberry 
Ken  Wolpers 
Ray  Klotz 


-  General  Dynamics 

-  McDonnell  Douglas 

-  Northrop  Corp 

-  Boeing  Aerospace  Co 

-  RCA  Corporation 

-  Honeywell  Avionics 

-  Lockheed  Missile 

and  Space 

-  Ball-Aerospace 

Systems  Div 
Space  Division  - 
AFSC 

-  Northrop  Electron¬ 

ics 

-  Boeing 

-  McDonnell  Douglas 

-  Honeywell 

-  Watkins  -  Johnson 


Letters  requesting  assistance  were  sent 
to  many  other  Directors  of  Quality  Assur¬ 
ance  throughout  Space  industry  in  search 
of  experts  in  the  eight  areas.  They  were 
asked  to  select  an  expert  in  a  specific 
area  to  work  with  the  committee  by  devel¬ 
oping  a  draft  standard. 


A  meeting  was  conducted  at  Martin 
Marietta,  Aerospace  Division,  Denver,  CO 
on  13  December  1979,  where  all  drafts  of 
the  workmanship  standards  were  reviewed, 
discussed,  critlzed,  and  many  changes  ham¬ 
mered  out.  Space  Division,  at  that  time, 
expressed  concern  and  reserve  at  develop¬ 
ing  a  standard  that  would  appear  to  be  al¬ 
lowing  discrepancies;  that  is,  developing 
standards  that  would  permit  some  crazing 
and  measling  and  using  jumper  wires  on  PC 
Boards.  To  Quality  Assurance  at  Space 
Division,  standards  of  this  type  would  be 
considered  as  falling  into  the  area  of  re¬ 
pair  procedures. 

Regardless  AIA/NSIA  intends  to  standard¬ 
ize,  wherever  possible,  so  that  industry's 
workmanship  (at  least  in  space  hardware) 
will  be  uniform.  The  Government  will  con¬ 
sider  each  standard  on  its  own  merit  prior 
to  converting  them  to  military  standards. 

After  rewriting  the  standards  another 
meeting  was  called  by  the  cochalrraan, 

Marty  Adams  and  Dick  Hannum.  (Hannum  had 
replaced  George  McGee.) 


403 


The  meeting  was  again  held  at  Rockwell, 

Seal  Beach  on  14  Feb  80.  The  committee 
members  had  done  their  work  well  because 
each  proposed  and  presented  standard 
which  had  been  previously  reviewed  by  them 
were  still  found  to  need  improvement.  At 
the  conclusion  of  the  meeting  the  drafts 
were  to  again  be  corrected.  For  the  hard 
work,  patience  and  dedication  I  would  like 
to  assure  credit  is  given  to  all  committee 
members  who  helped  develop  the  standards 
with  special  kudos  to  the' cochairmen. 

In  developing  workmanship  standards  for 
space  application  this  is  just  a  good 
start.  The  end,  I  am  sure,  will  never 
arrive  because  technology  advances  and  one 
innovation,  discovery  or  invention  succeeds 
the  next.  Space  Division  would,  however, 
like  to  develop  workmanship  standards  for 
parts,  assemblies  and  processes  known  to 
us  now.  Areas  that  have  not  been  addres¬ 
sed  (such  as  the  requirements  of  MIL-STD- 
454,  General  Requirements  For  Electronic 
Equipment,  MIL-E-8189,  Electronic  Equip¬ 
ment  Missiles,  Boosters  and  Allied  Vehi¬ 
cles,  General  Specification  For,  and 
MIL-E-4158,  Electronic  Equipment  for 
Aerospace  Ground  Equipment)  need  workman¬ 
ship  standards.  The  AIA/NSIA  Committee 
felt  that  at  this  time  It  was  more  impor¬ 
tant  to  complete  the  eight  areas  that  were 
selected. 

MIL-E-8189  references  many  specifica¬ 
tions  and  standards  for  parts,  materials, 
and  processes.  Each  one  of  the  referenced 
specifications  need  a  workmanship  stan¬ 
dard.  Until  we  have  standards  that  are 
used  industry-wide  there  will  continue  to 
be  variation  in  what  is  good  and  bad;  what 
is  acceptable  and  unacceptable.  Of  course, 
there  will  continue  to  be  variation  in 
workmanship  as  long  as  more  than  one  per¬ 
son,  company  or  line  peforms  the  work 
because  of  the  differences  in  craftsman¬ 
ship. 

I  define  craftsmanship  as  the  training, 
discipline  and  motivation  of  the  indivi¬ 
duals  accomplishing  the  workmanship. 

There  are  many  good  books  written  on  all 
three;  training,  discipline  and  motiva¬ 
tion.  Therefore,  all  I  will  add  is  -  the 
better  trained,  the  more  disciplined  and 
the  more  motivated  -  the  better  people 
performing  the  work  will  be  in  making  the 
craftsmanship  look  and  perform  the  same. 
Especially,  if  they  are  all  being  trained 


ana  are  working  from  the  same  specifica¬ 
tions  and  workmanship  standards.  It  is 
needless  to  say  that  without  management's 
endorsement  and  dedication  none  of  the 
controls  discussed  will  work. 

At  Space  Division  we  are  working  on 
ways  to  improve  discipline  and  motivate 
contractor's  personnel  and  that  way  is 
through  contractual  incentives.  We  are 
exploring  different  ways  to  include  in¬ 
centives  that  will  directly  effect  the 
individual  rather  than  or  as  well  as  the 
company.  Some  approaches  are  through 
award  fees  -  part  of  the  criteria  for  the 
successful  bider  will  be  a  plan  whereby 
the  individuals  working  on  the  program 
will  be  rewarded  for  certain  acts  and 
thus  motivated.  We  are  also  doing  some¬ 
thing  about  discipline  of  the  contractor's 
personnel. 

Through  our  extension  of  MIL-Q-9858A, 
SAMSO  STD  73-5B,  we  require  contractors 
to  maintain  a  training  program.  The 
training  program  requires  the  contractor 
to  certify  the  individuals  in  the  areas 
where  they  have  been  trained.  Recertifi¬ 
cation  after  certain  lengths  of  time  is 
also  required.  The  contractor  is  re¬ 
quired  to  identify  the  required  skills 
and  maintain  records  of  the  training, 
certifications  and  recertifications.  We 
believe  this  program  will  motivate  and 
discipline  personnel. 

It  is  the  Government's  policy  to  re¬ 
duce  or  elimiante,  where  possible,  speci¬ 
fications  and  standards . This  is  especially- 
true  where  specifications  include  by  ref- 
erence(the  treeing  effect)other  specifi¬ 
cations.  If,  however,  a  standard  for 
workmanship  can  be  developed  with  indivi¬ 
dual  requirements  and  the  requirements, 
in  turn,  are  complete  and  do  not  require 
other  specifications,  then  the  military, 
or  at  least  Air  Force,  will  publish  it. 

The  plans  are  to  develop  the  standard 
somewhat  like  MIL-STD-454,  with  separate 
requirements.  Unlike  MIL-STD-454  the 
workmanship  standards  will  be  totally  in¬ 
dependent  and  will  not  cause  other  speci¬ 
fications  or  standards  to  form  a  part  of 
the  workmanship  standard. 

The  need  is  clear  for  creating  a  uni¬ 
fied  workmanship  document.  The  task  to 
write  a  dynamic  document  that  is  easy  to 
add  to  or  delete  from  will  indeed  be  hard. 


PRESENTATION  -  APRIL  30,  1980 
MISSION  ASSURANCE  SYMPOSIUM 

Presenter: 

D.  M.  Verrastro  - 
Manager,  Employee/Labor  Relations 
Martin  Marietta  Aerospace/Denver  Division 

Do  you  feel  about  your  collective  bar¬ 
gained  agreements  like  some  people  feel 
about  their  marital  status-- 

(1)  I  don't  like  it 

(2)  I  can't  change  it 
but 

(3)  I  can't  help  dreaming  of  some¬ 
thing  better 

I  hope  today  to  address  the  viewpoint  that 
the  collective  bargaining  agreement  is  an 
instrument  that,  if  not  likeable  or 
embraceable,  can  be  at  least  be  tolerated 
and  worked  with,  can  be  changed  and  can  by 
bringing  understanding  be  something  better 
even  in  its  present  form  today-- 

"Management  has  not  set  high  enough  quality 
standards  and  what  they  have,  are  not  con¬ 
sistently  applied."  This  is  a  paraphrase 
of  a  recent  statement  made  on  national  TV 
news  show  by  a  prominent  authority  in  the 
field  of  labor  relations. 

What  this  person  was  saying  is  that  it  is 
management's  responsibility  to  set  qual¬ 
ity  standards  and  apply  those  standards 
uniformly.  What  message  could  be  more 
clear— 

Your  Job!  Do  it! 

No  way,  I  can't  do  it  -  I'm  not  permitted 
to  do  it  -  the  union  contract  interferes 
with  my  right  to  do  it  -  and  that's  the 
topic  of  our  discussion  during  the  next 
30  minutes.  I  submit  to  you  that  the 
union  contract  does  not  interfere  with  our 
right  to  operate  efficiently  and  set  high 
quality  standards. 

By  the  way,  the  prominent  authority  I've 
paraphrased  is  none  other  than  Douglas 
Fraser,  President  of  the  United  Auto 
Workers.  Mr.  Fraser  placed  the  responsi¬ 
bility  or  onus  on  management  when  asked  on 
the  "Good  Morning  America"  show— why  do 


U.S.  auto  makers  have  more  quality 
problems  than  foreign  manufacturers  with 
whom  they  compete. 

Mr.  Fraser  recognizes  that  we  have  not 
only  the  responsibility  to  establish 
quality  standards  but  I  suggest  to  all  of 
you  that  implicit  in  his  statement  is  the 
fact  that  management  has  the  authority  — 
Albeit  in  some  cases  modified  or  abrid¬ 
ged  —  to  set  quality  standards. 

Mr.  Fraser  indeed  was  suggesting  that 
collective  bargaining  equates  to  accom¬ 
modation  of  each  other's  problems, 

I.  We  in  management  have  a  problem  or  we 
feel  that  the  collective  bargaining  agree¬ 
ment  prevents  us  from  doing  our  job  -- 
if  that's  the  true  case,  then  we  have  no 
right  in  business  today,  or  we  must  change 
our  posture  (a  good  example  -  aluminum  - 
training  -  transfers) 

Collective  bargaining  equals:  accommo¬ 
dation  -  a  dictionary  definition  of 
accommodation  is:  an  adjustment  of  dif¬ 
ferences. 

The  union  has  the  right  to  bargain  for  its 
members  regarding  maintaining  and  improv¬ 
ing  working  conditions  —  that's  the 
union's  right  —  the  union  membership 
expects  the  bargaining  will  result  in  a 
maintenance  and  improvement  on  such  items 
as  wages  —  benefits.  I  don't  believe 
that  the  majority  of  union  members  have  a 
desire  to  put  management  that  is  the  com¬ 
pany  out  of  business  in  a  collective  bar¬ 
gaining  situation  --  although  this  can  and 
does  result  --  it  is  not  necessarily  an 
avowed  objective  of  union  members  and  or 
the  institution  union  to  do  this. 

Management  has  a  right  usually  expressed 
in  a  management  prerogatives  clause  to 
expect  a  fair  return  on  its  total  invest¬ 
ment  not  expressed  that  way  but  that's 
what  it  says  we  have  a  right  to  manage 
out  business  investment.  That  management 
usually  attains  that  objective  can  be 
readily  seen  by  any  review  of  the  fortune 
500/1000  companies  and  their  1979  profit¬ 
ability  margins  —  which  companies,  I  may 
add,  have  employees  who  are  represented 
by.  I'm  sure,  every  union  identified  as 
such  in  this  country. 

The  company  has  a  right  to  expect  that  the 
union  representing  him  will  at  the  very 


least  continue  to  support  his  efforts  to 
maintain  his  standard  of  living.  They 
have  a  right  to  expect  that  their 
employer/the  company  will  at  the  very 
least,  provide  a  safe  and  fair  place  of 
employment. 

The  employee  expects  that  his  company  and 
his  union  can  acconmodate  each  other. 

II.  Collective  Bargaining  Agreements 

Robert's  Dictionary  of  Industrial  Relations 
terms  defines  a  collective  bargaining 
agreement  in  these  words— a  contract  or 
mutual  understanding  between  a  union  and 
a  company  setting  forth  the  terms  and  con¬ 
ditions  of  employment  usually  for  a  speci¬ 
fic  period  of  time.  The  scope  and  cover¬ 
age  of  the  agreement  will  depend  on  the 
parties. 

The  U.S.  Supreme  Court  in  the  1961  steel¬ 
workers  trilogy  in  a  footnote  said— 

(Gulf  warrior) 

A  collective  bargaining  agreement  is 
an  effort  to  erect 

A  system  of  industrial  self 
government. 

Nothing  in  either  one  of  these  definitions 
suggests  that  collective  bargaining  is 
meant  to  or  sets  out  to  destroy  the  com¬ 
pany  --  or  its  products  or  service. 

Most  labor  agreements  in  a  manufacturing 
environment,  especially  in  our  aerospace 
environment  provide  for  a  quality  control 
organization  —  can  that  mean  that  Q.C.  is 
b  a  n  n  e  d/p  r  o  h  i  b  i  t  e  d—  of  course 
not  —  just  the  reverse  is  true  -  Q.C.  is 
encouraged  to  function. 

Why  doesn't  Q.C.  function  --  in  a  climate 
where  a  C.B.A.  exists? 

Because  --  seniority  provisions  cause  the 
company  to  lay-off  the  junior  —  last 
hired  --  but  highly  qualified  employee 
today  --  wasn't  that  same  provision  in  the 
labor  agreement  10  years  ago  --  when  the 
man  who  faced  lay-off  then  was  the  same 
man  we  wanted  to  protect,  but  now  we  say 
he  no  longer  can  do  this  job  —  is  that 
the  fault  of  the  labor  contract,  the  C.B.A. 
or  is  it  our  fault  for  not  providing  during 
the  last  10  years  the  training  necessary 


for  this  once-valued  employee  --  so  that 
they  could  function  in  today's  environ¬ 
ment  --  after  all,  we  got  10  years  out  of 
them  —  what  have  we  gotten  from  the  new 
hire??  Are  our  industry  certification 
programs  doing  the  job?  Whose  fault  is 
it  —  where  does  the  blame  lie?? 

Does  Q.C.  not  function  because  we  must 
promote  incompetence  by  virtue  of  the 
seniority  clause? 

We  in  Martin  have  gathered  information 
from  our  competitors  whenever  we  prepare 
for  collective  bargaining  and  I  don't 
recall  comparing  any  seniority  provision 
in  the  aerospace  industry  that  stated  — 
seniority  shall  be  the  only  selection  cri¬ 
teria  in  determining  promotions.  Usually 
the  clause  is,  and  I'll  paraphrase  --  all 
things  being  equal — that  is  ability  to 
perform  the  job,  then  seniority  will 
govern — 

If  your  agreement  says  seniority  is  the 
sole  criteria  for  all  promotions,  then 
you  do  have  a  problem  --  but  it  is  some¬ 
thing  that  can,  if  you're  truly  operating 
at  a  loss  because  of  this  situation  --  you 
can  change  and  more  and  more  manage¬ 
ments  are  stepping  up  to  the  collective 
bargaining  table  and  saying,  accommodate 
a  very  real  problem  and  more  and  more 
unions  are  listening  and  favorably 
responding  — 

Do  job  descriptions  limit  quality  -- 
answer;  again,  is  an  emphatic  no  quality 
is  permitted  -- 

The  union  member  wants  a  job  that  he  can 
take  pride  in.  Yes,  collective  bargain¬ 
ing  agreement  can,  and  sometimes  does, 
stifle  quality  performance  --  the  union  is 
a  political  entity  that  tends  to  the  mid¬ 
dle  of  the  road  —  but  the  union  contract 
does  not  snuff  out  quality  --  the  union 
member  does  not  want  quality  snuffed 
Out  —  he/she  —  they  —  want  to  continue 
to  be  recognized  for  their  achievements; 
however  small  that  achievement  may  be;  for 
example,  does  anybody  here  have  a  union 
Contract  which  would  prohibit  an  employee/ 
union  member  from  going  to  the  customer's 
location  and  seeing  what  the  end  product 
looks  like  —  how  it  functions  --  titan 
program  —  select  various  in-unit  types  — 
flights  to  the  cape,  western  test  range, 
etc. 


Quality  like  safety,  is  an  attitude  -- 
create  the  proper  climate,  with  or  without 
a  labor  agreement  recognize  the  individual, 
remove  the  obstacles  and  the  way  we  per¬ 
ceive  a  labor  agreement  can  be  a  large 
real  obstacle  to  obtaining  performance, 
clear  up  our  thinking  —  correct  past 
practices  --  and  they  can  be  corrected 
and  we  can  accommodate  quality  in  the 
work  place,  even  in  a  collective  bargain¬ 
ing  environment. 

Let's  accept  Mr.  Fraser's  challenge  and 
make  collective  bargaining  work  to  achieve 
quality. 
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V 


IS  IT  REALLY  WORKMANSHIP? 

by  Paul  L.  Dalton 
Program  Manager,  Quality 
Martin  Marietta  Aerospace 
Denver  Division 


INTRODUCTION 

Workmanship  is  defined  as  the  art, 
skill,  or  technique  of  a  workman. 
Poor  workmansliip  is  then  the  lack 
of  these  attributes.  In  speaking 
of  workmanship  in  the  aerospace 
industry  we  may  ask  ourselves, 

"Are  our  workmen  generally  under- 
skilled  or  undermotivated?"  If  the 
answer  is,  "No",  then  what  is  our 
problem? 

To  answer  this  question  we  need  to 
examine  the  environment  in  which  we 
work.  While  it  is  true  that  our 
work  environment  is  very  demanding 
(probably  more  so  than  other  indus¬ 
tries)  and  the  environment  is  being 
changed  by  trends  towards  auto¬ 
mation  and  away  from  hand  skills, 
these  are  not  areas  in  which  our 
workmanship  problems  can  be  address¬ 
ed.  Improvement  can  only  be  real¬ 
ized  when  we  understand  how  defects 
are  produced.  The  assumption  that 
defects  are  man  related  is  often 
not  true.  Frequently,  defects  are 
producted  by  Inadequate  or  unfor- 
glveable  processes  and  designs. 

When  this  is  true,  defects  are  high¬ 
ly  probable  from  the  beginning  con¬ 
cept  of  hardware. 

Significant  Improvement  can  only  be 
realized  when  we  understand  how 
defects  are  being  produced  and  how 
to  eliminate  them.  Today  we  will 
discuss  several  tools  that  I  think 
can  be  effectively  applied  toward 
this  end. 

TOOLS  FOR  DEFECT  UNDERSTANDING  AND 
PREVENTION 

The  tools  that  we  will  discuss  here 
are  not  new  but  tools  that  have 
been  around  for  a  long  time  that 
need  to  be  dusted  off  and  applied 
with  new  dedication  and  exactness. 


A.  Design  Review 

The  specific  objective  of  design 
review  is  to  come  up  with  a  de¬ 
sign  that  can  be  fabricated, 
tested  and  flown  as  problem  free 
as  possible.  This  can  be 
achieved  by  finding  and  elimin¬ 
ating  problems  before  they  occur. 
Problems  fit  into  two  categories, 
new  problems  and  old  problems. 

New  problems  are  those  intro¬ 
duced  into  the  design  by  new 
technologies.  Old  ones  are 
those  that  have  occurred  before 
on  prior  programs.  The  latter 
normally  form  a  significant 
portion  of  the  population  of 
problems  encountered  which 
serves  to  stress  the  importance 
of  utilizing  experience,  not  only 
your  own,  but  the  experience  of 
others  who  have  worked  on  simi¬ 
lar  programs. 

When  assessing  a  design,  it  is 
necessary  to  allow  for  variations 
to  norm.  The  only  way  to  do  this 
is  to  be  very  conservative  in 
estimates  of  capability.  Get  as 
much  margin  in  planning,  process¬ 
es,  drawings  and  parts  as  possi¬ 
ble.  Margin  can  be  obtained  In 
design  by  expanding  tolerances. 
Margin  can  also  be  obtained  by 
extra  care  in  handling  or  in 
fabrication  tooling  that  keeps 
the  hardware  relatively  stress 
free. 

Problem  Examples; 

1.  Old  Problem:  Forward  and 
aft  OMNI  antennas  experi¬ 
enced  dipole  and  crossover 
wire  fatigue  failures  during 
qualification  vibration 
tests.  Yes,  I'm  aware  that 
qualification  tests  are  to 
confirm  the  inherent  design 
capability,  but  the  key 
word  is  to  "confirm".  Prior 
experience  definitely  pointed 
to  the  need  to  tie  flimsy 
things  like  this  down.  A 
problem  could  have  been 
prevented . 

2.  New  Problem:  Steerable 
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Horizon  Crossing  Indicator 
experienced  bolometer  sep¬ 
aration  from  heat  sink  dur¬ 
ing  acceptance  thermal  cycl¬ 
ing.  The  coefficient  of 
thermal  expansion  between 
two  materials  (selenium  - 
germanium)  was  too  great, 
setting  up  stresses  and 
causing  the  selenium  to 
crack.  The  design  was  not 
new  but  the  temperature 
cycles  It  was  subjected  to 
were  in  excess  of  what  It 
had  ever  been  subjected  to 
before.  Another  problem 
that  could  have  been  pre¬ 
vented  . 

It's  true  that  we  always  have 
20/20  hindsight,  but  this  just 
points  up  the  need  to  be  tech¬ 
nically  Inquisitive  and  alert. 
Obvious  problems  have  a  habit 
of  slipping  by  and  are  expensive 
and  embarrassing  when  allowed 
to  become  evident  as  fabrication 
and  test  progress. 

A  point  that  also  must  be  made 
is  that  design  review  Is  not 
just  PDR/CDR.  Design  review  is 
not  something  that  Is  performed 
in  two  days  by  sitting  down 
with  a  set  of  drawings,  but  in¬ 
volves  detail  interfaces  with 
designers  and  others,  both  in- 
house  and  subcontractors.  The 
review  must  start  as  the  prop¬ 
osal  Is  being  prepared  and  is 
really  not  complete  until  after 
launch,  although  a  maximum  eff¬ 
ort  must  be  focused  to  eliminate 
all  possible  problems  from 
designs  prior  to  release  for 
procurement  or  fabrication. 

B.  Process  Reviews  and  Validations 

Processes  can  significantly  re¬ 
duce  design  margin  and  detract 
from  the  workman's  ability  to 
produce  an  acceptable  product. 
Presented  here  are  a  couple  of 
typical  process  type  problems 
that  fit  Into  the  old  and  new 
categories. 

Problem  Examples: 


1.  Old  Problem:  High  thrust 
engine  thrust  chamber  valve 
failed  pull-in  and  drop-out 
current  tests.  Water  flush 
induced  abnormal  drag  be¬ 
tween  polished  steel  surface 
on  plunger  and  polished 
steel  surface  on  valve  wall. 
Alcohol  flush  eliminated 
problem.  This  was  another 
case  of  falling  back  into  a 
known  trap. 

2.  New  Problem;  Initial  wave 
soldering  of  PWB's  with 
Macdermld  Macu-Mask  solder 
resist  with  traces  equal  to 
or  greater  than  0.050  re¬ 
sulted  in  cracking  of  resist 
and  subsequent  entrapment  of 
liquid  contaminant.  The 
process  had  been  validated 
on  boards  with  narrower 
traces  without  a  problem. 

It  can  be  seen  that  the  afore 
listed  problems  should  have 
been  prevented.  The  examples, 
and  many  others  any  of  us  could 
think  of,  stress  the  need  to 
carefully  evaluate  each  step  of 
the  process.  A  technique  of 
process  evaluation  might  consist 
of  a  series  of  questions  such 
as  the  following. 

1.  What  effects  will  it  have  on 
the  materials  being  applied? 

2.  Does  it  Involve  only  oper¬ 
ations  within  normal  work¬ 
man  skills,  or  are  there 
difficult  operations  that 
require  special  training? 

3.  Are  there  any  new  or  state- 
of-the-art  applications  or 
operations? 

4.  Are  inspections/tests  clear¬ 
ly  defined  and  are  they 
completely  adequate  to 
assure  that  the  article  in 
all  respects  meets  the 
design  requirement? 

5.  What  prior  problems  are 
known  to  exist  with  this 
type  process? 
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6.  Have  all  the  pitfalls  been 
recognized  and  eliminated? 

Once  the  review  effort  is  com¬ 
plete,  the  next  step  is  process 
validation.  This  involves 
gathering  the  Process  Engineer, 
Quality  Engineer,  and  manufac¬ 
turing/fabrication  personnel 
together  in  the  actual  work 
environment  (location  and  hard¬ 
ware)  to  confirm  that  each  step 
that  is  written  on  the  paper  is 
clearly  understandable  and 
Implementable  in  the  fabrication 
environment . 

Software  Reviews/Validation 

For  the  purpose  of  this  paper, 

I  don't  want  to  write  a  treatise 
on  software  (it's  a  subject  all 
by  itself) ,  but  I  do  want  to 
point  to  the  need  to  exercise 
the  same  type  tools,  the  same 
type  vigilance  and  control  on 
software  that  we  do  on  hardware. 
Software  errors  are  time  consum¬ 
ing  and  expensive,  and  most  that 
we  experience  are  preventable. 

The  problems,  as  in  hardware, 
can  start  anywhere  in  the  devel¬ 
opment  process.  Post  analysis 
of  software  problems  has  indic¬ 
ated  that,  most  frequently,  they 
are  traceable  to  the  following 
basic  development  process  weak¬ 
nesses. 

1.  Lack  of  requirements  defini¬ 
tion. 

2.  Inadequate  designs  from 
which  to  code. 

3.  Complex  coding  techniques 
making  it  difficult  to  see 
errors. 

4.  Lack  of  adequate  document¬ 
ation  to  trace  actions  and 
lack  of  modularity  and  main¬ 
tainability. 

To  be  effective.  Quality  must 
focus  its  effort  on  each  phase 
of  software  development,  coding, 
testing,  configuration  control 


and,  last  but  not  least,  fail¬ 
ure  analysis  and  corrective 
action.  An  effective  failure 
analysis  and  corrective  action 
effort  that  inists  that  the 
cause  of  each  problem  is  deter¬ 
mined  and  assure  that  corrective 
action  is  assigned  is  equally 
as  Important  in  software  as  it 
is  in  hardware. 

D.  Training 

Most  of  us  in  the  Industry  have 
training  programs  because  we 
know  that  they  are  necessary  to 
assure  that  worker  skills  are 
maintained  at  an  acceptable 
level.  However,  I  believe  that 
at  times  there  may  be  a  tendency 
to  allow  the  training  programs 
to  kind  of  take  their  own  course 
and,  thereby,  tend  to  lose  their 
relevancy.  To  be  effective, 
training  must  be  specifically 
directed  towards  the  work  that 
is  to  be  performed.  This  in¬ 
volves  verifying  performance 
capability  of  workers  on  actual 
hardware  and  under  typical  shop 
work  conditions.  This  means 
that  you  have  to  stay  on  top  of 
your  training  program  all  the 
time  to  be  able  to  recognize 
when  needs  are  changing.  It  is 
also  very  important  to  continu¬ 
ously  monitor  work  results  to 
identify  specific  workmanship 
problems  and  training  inadequac¬ 
ies  . 

Another  very  valuable  tool  that 
can  be  utilized  to  find  problems 
leading  to  workmanship  deficien¬ 
cies  is  to  talk  to  the  workmen. 
The  people  that  do  the  work  day 
in  and  day  out  know  where  they 
are  encountering  difficulties 
and  if  they  tell  you  that  your 
process  or  design  stinks,  you 
better  believe  them.  The  prob¬ 
lems  discovered  must  then  be 
worked  in  some  way.  If  it's  not 
feasible  to  change  the  design 
or  process,  certainly  the  manu¬ 
facturing  techniques  or  training 
needs  require  investigation. 


E.  Failure  Analysis  and  Corrective 
Action 

The  first  thing  I  would  like  to 
mention  here  is  attitude.  The 
attitude  that  says  a  certain 
number  of  failures  is  Inevitable. 
Yes,  I  know  what  happens  in  the 
real  world.  But  to  start  with 
an  attitude  that  some  failures 
are  Inevitable  means  that  I 
start  not  shooting  for  100%. 

I  believe  that  this  tends  to 
lull  everybody  into  a  ho-hum 
approach  to  problem  prevention. 

We  need  to  expect  success  and 
have  a  real  conviction  going  in 
that  all  the  holes  are  closed. 

Then,  if  a  failure  should  occur, 
you  leap  right  on  top  of  it  to 
find  the  hole  that  was  not 
closed.  Each  failure  or  problem 
must  be  Investigated  to  Isolate 
the  root  cause  and  corrective 
action  taken  to  prevent  recurr¬ 
ence.  Many  times  I  have  become 
involved  with  problems  where  it 
was  stated  that  nobody  had  any 
idea  what  to  do  for  corrective 
action.  Almost  without  excep¬ 
tion,  it  was  discovered  that  the 
real  problem  was  that  the  cause 
of  the  problem  had  not  been 
defined.  Corrective  action 
becomes  very  evident  when  it  is 
understood  what  (cause)  must  be 
prevented . 

To  provide  a  typical  model  for 
evaluation,  I  have  taken  signifi¬ 
cant  problems  from  a  typical 
spacecraft  program  and  categor¬ 
ized  them  by  causes  into  design, 
process  and  Workmanship.  It  is 
recognized  that  the  evaluation 
of  what  fell  into  which  category 
was  somewhat  arbitrary,  but  the 
figures  should  be  representative. 
The  relative  percentages  for  the 
significant  problems  were  54% 
design,  20%  process,  and  26% 
workmanship . 

This  was  a  one  of  a  kind  type 
program  that  went  directly  into 
production  as  opposed  to  one 
where  there  was  significant 
development  effort.  This  may 


tend  to  explain  the  relatively 
higher  number  of  design  problems. 
Also,  another  contributing  factor 
is  that  only  significant  problems 
were  selected.  These  were  the 
problems  that  commanded  signifi¬ 
cant  attention  either  from  a 
technical,  schedule,  or  cost 
impact.  Regardless  of  whether 
the  ratios  seem  quite  right,  it 
clearly  points  to  the  need  to 
focus  attention  on  all  areas  - 
all  were  significant  contribu¬ 
tors.  The  sample  taken  and  the 
examples  given  also  included 
many  problems  that  occurred  at 
subcontractors,  so  it  clearly 
indicates  that  the  actions  taken 
must  not  only  be  focused  on  in- 
house  activities,  but  on  sub¬ 
contractors  as  well. 

Specification  Problems 

I  don't  have  a  tremendous  axe  to 
grind  in  this  area,  but  some  prob¬ 
lems  did  occur  to  me  that  we  have 
encountered  before. 

The  first  problem  is  with  MIL-S- 
45743  which  limits  flux  usage  to 
RMA  only.  This  is  fine  for  most 
solder  applications,  but  there  are 
cases,  certain  lead  materials,  etc., 
that  would  react  better  to  more 
active  fluxes. 

Another  case  that  comes  to  mind  is 
MIL-C-46058  which  requires  conformal 
coating  other  than  the  conformal 
coat  we  normally  use.  This  could 
contribute  to  a  workmanship  problem 
if  we  were  forced  to  change  due  to 
the  new  process.  So  far,  we  have 
been  able  to  get  an  exception  for 
every  program  that  allows  us  to  use 
our  normal  conformal  coat. 

The  more  common  problem  that  I  have 
run  into  in  this  area  is  excessive 
interpretation  of  well  meaning  specs. 
A  typical  example  here  is  with  MIL- 
S-52779  which  requires  control  of 
nondeliverable,  as  well  as  deliver¬ 
able  software.  The  requirement  is 
wide  open  for  interpretation  and 
if  interpreted  literally,  will 
involve  a  lot  of  effort  and  expense 
that  will  not  have  any  positive 
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impact  on  product  quality. 


The  problems  then  fall  into  two 
categories.  In  the  first  case,  if 
the  problems  are  not  noted  and 
worked,  you  have  a  negative  impact 
on  product  quality,  and  in  the  sec¬ 
ond  case,  you  have  energies  and 
attentions  diverted  from  significant 
to  insignificant  tasks. 

Conclusion 


One  conclusion  that  we  can  definite¬ 
ly  draw  is  that  it's  not  going  to  be 
effective  to  address  workmanship  as 
a  general  problem.  It's  a  many 
faceted  problem  that  must  be  add¬ 
ressed  specifically  towards  a  known 
problem  area  to  realize  improvement. 

I  think  that  motivational  programs 
are  beneficial,  especially  those 
that  let  workers  know  that  they  are 
an  Important  part  of  the  team,  but 
it's  not  a  panacea.  As  a  matter  of 
fact,  motivational  efforts  directed 
towards  workmanship  improvement  with¬ 
out  a  determined  effort  in  evidence 
to  eliminate  defects  can  be  negative. 
All  we  do  is  convince  the  workmen 
that  we  don't  know  what's  going  on. 

A  number  of  tools  have  been  discussed 
here  today,  and  it's  my  conviction 
and  experience  that  they  can  be 
effectively  applied.  I  believe  that 
tools  of  this  nature,  rigorously 
and  exactly  applied,  are  not  only 
needed  but  are  necessary  to  perform 
in  the  aerospace  work  environment. 


MANUFACTURING  READINESS  FOR  PRODUCTION 
Richard  A.  Sipe 

lUS  MANUFACTURING  MANAGER 
BOEING  AEROSPACE  CO. 

Manufacturing  readiness  for  production 
starts  early  in  the  design  phase  of  a  pro¬ 
gram  with  integration  of  design  and  produc¬ 
tion  producibility  analysis  activities. 
Increased  emphasis  is  being  placed  by  the 
Government  on  production  management  early 
involvement  in  the  product  acguisition 
cycle.  Special  attention  is  being  demanded 
to  decrease  production  costs  prior  to  as 
well  as  during  production.  To  meet  this 
objective  planned,  explicit,  and  timely 
assessments  of  the  production  management 
implications  and  production  risks  are 
necessary  from  the  beginning  of  the  acqui¬ 
sition  cycle  throuqh  the  decision  to  go 
into  production. 

Mi 1-Std-l 528,  'Production  Management," 
describes  the  Air  Force  requirements  for 
an  effective  production  management  system. 
This  Mil-Std  was  written  to  be  applied  in 
total,  or  tailored  to,  the  specific 
requirements  of  each  contract  based  on 
program  technical  complexity,  production 
risk,  production  cost,  scope  of  production 
operations,  and  potential  for  program 
impact  due  to  production  operations. 
Implementation  of  Mil-Std-1528  typically 
requires  documentation  (a  formal  produc¬ 
tion  plan)  and  the  conduct  of  formal  pro¬ 
duction  readiness  reviews  prior  to  transi¬ 
tion  from  full  scale  development  to 
production.  The  following  discussion  is 
drawn  from  my  Boeing  manufacturing  opera¬ 
tions  experience  with  several  programs, 
but  particularly  from  lUS. 

The  success  of  a  product  such  as  lUS  is 
established  during  the  design  phase  of  a 
program.  For  example,  during  the  lUS 
concept (val idation)  phase,  Boeing  designed 
the  upper  stage  vehicle,  developed  and 
tested  certain  high  risk  hardware,  and  con¬ 
ducted  many  major  tradeoff  producibility 
analyses.  Manufacturing's  key  role  during 
this  early  phase  involved  fabrication  and 
assembly  of  developmental  test  hardware, 
initiation  of  a  producibility  program  for 
design  analysis  and  supportive  trade 
studies  on  alternate  design  concepts,  par¬ 
ticipation  in  subcontractor  source  selec¬ 
tion  surveys,  preparation  of  preliminary 
make/buy  plans,  and  preparation  of  a 
preliminary  production  plan. 


Certainly  a  key  activity  to  the  development 
of  a  good  production  plan  is  the  determina¬ 
tion  and  optimization  of  the  product's 
producibility.  Producibility  considerations 
during  preliminary  planning  include  economic 
evaluation  of  alternative  configurations, 
materials,  methods,  processes,  technology, 
tooling,  test  equipment  and  procedures, 
sequence  of  processes,  factory  layout  and 
flow,  lot  size,  cyclic  demand,  packaging 
and  handling,  inspection,  manual  versus 
computer  part  processing,  and  capability 
and  capacity  versus  outside  source  develop¬ 
ment.  These  considerations  also  include 
review  of  drawings,  specifications,  test 
procedures,  and  tool  and  production  plan¬ 
ning  to  verify  that  all  necessary  informa¬ 
tion  is  included  for  the  craftsmen  in  the 
factory  to  fabricate,  assemble,  install, 
and  test  parts  and  assemblies  within 
program  requirements.  This  activity 
requires  a  coordinated  effort  between  the 
designer  and  manufacturing  engineer  respon¬ 
sible  for  a  hardware  package.  Theirs  is  a 
continuing  relationship  from  concept  through 
final  determination  of  a  production  configu¬ 
ration.  They  are  supported,  as  necessary, 
by  Materiel,  Manufacturing  Technology, 
Industrial  Engineering,  Quality  Assurance, 
and  Factory  organizations  to  assure  that 
all  elements  affecting  a  cost  effective, 
reliable  end  product  are  identified  and 
incorporated. 

While  producibility  analysis  is  a  signifi¬ 
cant  activity  in  reducing  production  risk, 
it  is  not  always  a  precise  science.  It  is 
more  of  an  art,  where  perseverance  and  just 
plain  hardnosed  experience  are  the  key  to 
success.  To  explain  what  I  mean,  let  me 
relate  the  PDU  chassis  story.  The  power 
distribution  unit  (PDU),  one  of  the  lUS 
electrical  power  distribution  system  elec¬ 
tronic  boxes,  was  design-specified  as  a 
welded  chassis  assembly  constructed  of  many 
6061  aluminum  detail  parts.  Producibility 
analysis  of  the  PDU  chassis  resulted  in 
the  following: 

0  Reduction  in  part  count  by  32  (44-12) 

0  Reduction  in  manufacturing  flow  time  by 
48  mandays  (90-48) 

0  Reduction  in  linear  inches  of  weld  by 
375  inches  (many  multiple  passes) 

0  Weld  repair  drastically  reduced  through 
use  of  2219  aluminum  material 
0  Heat  treat  eliminated  (age  only) 

0  Weld  engineer  surveillance  virtually 
eliminated 

0  Factory  labor  manhours  per  unit  were 
significantly  reduced 
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0  Product  quality  and  appearance 
qreatly  enhanced 

This  story  makes  the  point  that  craftsman¬ 
ship  in  the  factory  must  be  backed  up  with 
craftsmanship  in  design,  producibility 
analysis,  and  production  planning  to 
achieve  optimum  end  product  results. 

Not  all  hardware  in  any  system  is  made  by 
the  prime  contractor,  so  attention  must  be 
paid  to  the  subcontractors  and  their  pro¬ 
ducibility  analyses  and  documentation. 
Engineering,  Manufacturing,  and  Quality 
Assurance  provide  technical  support  to  the 
subcontractors  in  their  producibility 
evaluations. 

As  noted  earlier,  one  of  the  Government 
initiatives  promulgated  through  Mil-Std- 
1528  is  the  requirement  for  formal  production 
plans.  Documentation  portrays  "Methods  and 
concepts  for  employing  facilities,  tooling, 
and  manpower  resources  of  the  contractor 
and  subcontractors.  It  reflects  all  time- 
phased  production  actions  required  to 
produce,  test,  inspect,  and  deliver  accep¬ 
table  contractual  end  items  on  schedule  at 
minimum  cost." 

The  production  plan  usually  is  comprised 
of  the  following  sections: 

I  Manufacturing  Organization 

II  Make  or  Buy 

III  Subcontracting 

IV  Resources  and  Manufacturing  Capability 

V  Production  Planning 

Detail  requirements  for  each  section  of  the 
production  plan  can  be  found  in  Government 
Data  Item  Description  DI-P-3460.  The 
significance  of  the  production  plan  is  that 
it  provides  a  detailed  "road  map"  from 
drawing  to  completed  hardware.  It  incor¬ 
porates  the  results  of  producibility  analy¬ 
sis  and  trade  studies.  It  is  submitted  to, 
and  approved  by,  the  Government  and  updated 
during  various  program  phases.  Properly 
done,  a  production  plan  is  Manufacturing 
Man's  bible. 

Using  the  production  plan  as  a  baseline, 
verify  that  your  homework  has  been  done  and 
your  planning  is  complete.  This  requires 
the  contractor  and  the  Government  to  conduct 
indepth  reviews.  As  an  example,  I  will 
describe  how  Boeing  and  Space  Division/NASA 
conducted  reviews  of  lUS  readiness  for 
production. 

DOD  Instruction  5000,28  describes  the 
objective  of  a  production  readiness  review 
(PRR)  as  follows: 


"To  verify  that  production  design,  planning, 
and  associated  preparations  for  a  system 
have  progressed  to  the  point  where  a  pro¬ 
duction  commitment  can  be  made  without 
incurring  unacceptable  risks  of  breaching 
thresholds  of  schedule,  performance,  cost, 
or  other  established  criteria." 

The  criteria  for  conduct  of  PRRs  are  pro¬ 
vided  in  AFSCP84-2  and  overall  production 
management  evaluation  approach  is  documented 
in  AFSCP84-3.  These  instructions  deal  with 
functions  that  are  highly  specialized  and 
subject  to  differing  interpretations,  defi¬ 
nitions,  and  understandings.  Many  program/ 
functional  groups  in  Sovernment/contractor 
organizations  are  impacted.  Misunder¬ 
standing  of  interpretations  and  requirements 
can  be  avoided  by  very  early  joint  partici¬ 
pation  by  contractor  and  the  Government  in 
developing  a  production  readiness  review 
plan. 

The  Boeing/Government  team  who  conducted 
11  PRRs  at  subcontractors  supplying  mission 
critical  hardware  to  lUS  were,  for  the  most 
part,  the  same  team  that  conducted  the 
Boeing  PRR.  During  the  subcontract  PRRs, 
Boeing  conducted  the  review  with  Government 
team  members  acting  as  advisors.  During 
the  Boeing  PRR,  the  roles  were  reversed 
with  Government  team  members  conducting  the 
review  and  Boeing  team  members  providing 
the  information.  This  is  an  effective  way 
of  developing  an  integrated  industry/Govern- 
ment  team  early  in  the  program.  It  pro¬ 
vides  a  mutual  understanding  of  system  ele¬ 
ments,  functions,  and  interfaces  and  leads 
to  an  efficient  and  economical  system-level 
review. 

I  believe  I  can  offer  a  few  "Lessons 
Learned"  which  may  help  in  demonstrating 
manufacturing  readiness  for  production. 

Do: 

0  Understand  what  PRRs  are  all  about  and  the 
significance  the  customer  places  on  them. 

0  Posture  yourself  early  enough  to  be  ready. 

0  Review  and  prepare 
0  Agenda 

0  PRR  criteria  and  ground  rules 
0  Program's  themes/messages 
0  A  single  thread  story 
0  Understand  all  other  elements,  functions, 
and  interfaces 

0  Reflect  a  single  program  team  approach. 

0  Use  the  production  plan  as  baseline  for 
the  PRR. 

0  Be  responsive  to  the  customer  and  under¬ 
stand  his  requirements  at  all  times. 


0  Prompt  a  response-oriented  session  where 
the  customer  will  discuss  his  concerns. 

0  Balance  showmanship  with  substance. 

Don't: 

0  Surprise  the  customer  in  a  PRR. 

0  Forget  to  be  prepared  to  address  problems 
encountered  during  developmental  phases 
and  do  show  how  they  were  solved  before 
production. 

0  Indicate  everything  is  peaches  and  cream; 
risks  do  exist;  therefore, 

0  Identify  risks 

0  Show  good  abatement  plans,  and 
0  Show  that  risks  are  controlled. 

0  Forget  to  follow  up  on  questions,  comments, 
or  last  PRR  issues. 

0  Be  caught  on  Mil-Stds  (1695,  1520A,  1567, 
1528,  etc.). 

0  Develop  a  proposal  one  way  and  really 
plan  to  do  it  differently. 

0  Try  to  con  the  customer;  they  are  as 
smart  as  you  are  and  have  probably  been 
through  this  more  than  you. 

0  Hipshoot  answers  -  get  back  with  specific 
details. 

0  Give  impromptu  demonstrations;  if  a 
demonstration  is  worth  giving,  it  is 
worth  preparing  for. 

In  summary,  three  recommendations  are 

offered; 

0  The  Contractor/Government  must  mutually 
define  the  manufacturing  readiness 
criteria  early  in  the  program. 

0  Utilize  resident  Government  agencies 
(AFPRO,  DCAS,  NASA)  in  the  planning  and 
conduct  of  production  readiness  reviews 
at  both  the  prime  contractor  plant  and 
subcontractor  plants. 

0  Contractor/Government  must  both  do  your 
homework;  know  the  production  plan,  and 
the  design  and  specification  requirements 
before  reviews  are  conducted. 
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CRAFTSMANSHIP 
MANUFACTURING  READINESS 


Ed  Huston 
BMU/MNCPB 

NORTON  AFB,  CALIFORNIA 

Increased  emphasis  is  being  placed  by  the 
government  on  production  management  early 
in  the  acquisition  cycle;  More  attention  is 
being  paid  by  the  government  to  specific  pro¬ 
grams  to  decrease  production  costs  prior  to 
and  during  production.  Summary  of  the  dis¬ 
cussion  is  as  follows: 

A.  Engineering/Product  Design 

Engineering  and  product  design  are  reviewed 
and  evaluated  to  assure  that: 

1)  Producibility  analysis  and  production 
capability  were  accomplished. 

2)  Cost  effectiveness  studies  were  accom¬ 
plished. 

3)  Risk  analysis  accomplished  and  new 
production  methods  or  techniques 
determined. 

4)  Contractor  made  maximum  use  of 
military  standard  components,  parts, 
and  processes  in  the  design. 

B.  Manufacturing  Planning 

Rescue  production  capabilities  are  deter¬ 
mined  which  can  be  used  to  minimi  2®  un¬ 
foreseeable  delays  or  expansion  of  sched¬ 
ules.  Production  organization  is  reviewed 
to  determine  if  schedule  adjustments  can  be 
accommodated.  Industrial  engineering 
responsibilities  and  authority  for  in-plant 
layout  and  flow  processes  is  evaluated. 
Actual  observation  of  manufacturing  and 
assembly  operations  is  accomplished  to 
assure  efficient  and  effective: 

1)  Machine  scheduling  and  loading 

2)  Machine  and  tooling  setup 

3)  Manpower  utilization 

4)  Minimum  scrap  or  rework  items 

5)  Adequate  work  instructions  for  the 
workers 


Utilization  of  latest  manufacturing  methods 
and  techniques  are  determined  along  with 
evaluating  the  adequacy  of  contingency 
manufacturing  plans. 

C.  Production/Manufacturing  Operations 

Production  and  manufacturing  operations 
are  reviewed  and  evaluated  for  the  follow¬ 
ing  items: 

1)  Evaluate  contractors  prpgram  for 
assuming  adequacy  of  production  skills. 

2)  Determine  extent  that  production 
management  is  responsive  to  new  and 
advanced  production  management 
techniques  and  advanced  production 
processes  and  techniques. 

3)  Review  clean  room  facilities. 

4)  Determine  if  automated  production 
procedures  provide  adequate  information 
regardii^  Ihe  status  and  location  of 
work-in-process,  allocation  versus 
actual  time,  and  other  useful  infor¬ 
mation. 

5)  Determine  if  manufacturing  has  a 
systematic  program  for  calibration  of 
production  process  gauges. 

D.  Production/Manufacturing  Methods  and 
Processes 

Specified  methods  or  processes  are 
evaluated  to  determine  if  trial  runs  prior 
to  initiation  of  production  has  been  ac¬ 
complished.  Efficiency  standards  must 
be  maintained  for  each  machine,  depart¬ 
ment,  or  process.  Actions  taken  to  pro¬ 
vide  for  differences  in  tooling,  material, 
and  process  control  and  operator  effect¬ 
iveness  in  transitioning  from  full-scale 
development  to  production  is  reviewed. 

The  methods  and  procedures  to  be  used  to 
Identify  discrepancies  between  scheduled 
and  actual  production  are  reviewed. 

E.  Tooling  and  Test  Equipment 

The  quantity,  type  and  cost  of  special  tool¬ 
ing  and  test  equipment  used  for  assembly, 
fabrication  and  test  is  evaluated.  Also 


the  planned  use  of  numerical  control  mach¬ 
ines  for  effectiveness  of  operations  is  re¬ 
viewed  and  evaluated.  The  prime  con¬ 
tractors'  evaluation  of  his  major  sub¬ 
contractor's  tooling  plans,  concept,  and 
capabilities  are  reviewed.  Ensure  that 
tooling  procedures  and  contingency  plans 
have  been  prepared  and  are  adequate  to 
assure  uninterrupted  and  continued  pro¬ 
duction. 

F.  Facilities  and  Equipment 

Determination  if  more  modem  equipment 
is  justified  to  reduce  machining  costs,  pro¬ 
vide  less  downtime,  and  lower  maintenance 
costs.  Planned  requirements  and  use  of 
numerical  and  tape  controlled  and  other  high 
cost  automated  equipment  is  reviewed. 

Also  the  plaimed  equipment  is  reviewed  to 
determine  producibility  of  acceptability  of 
parts  to  support  the  production  rate. 

G.  Subcontracts 

The  contractor's  subcontract  stmcture  is 
reviewed  to  determine  if  a  periodical  re¬ 
examination  is  accomplished  to  assure 
that  existing  subcontractors  remain  most 
economical  and  advantangeous  to  the  pro¬ 
gram.  Controls  employed  by  subcontract 
administration  for  monitoring  and  fore¬ 
casting  unsatisfactory  conditions  relating 
to  scheduled  deliveries  are  evaluated. 

H.  Quality  Assurance 

Design  review  process  is  evaluated  to 
determine  if  Quality  Assurance  has  parti¬ 
cipated  by  assessing  the  adequacy  of  exist¬ 
ing  equipment  and  procedures  to  control 
product  quality.  Procedures  for  control  of 
special  processes  and  systems  are  review¬ 
ed  for  adequacy  and  the  quality  controls 
established  for  materials  treatment  and 
processes  to  be  used  in  production  are 
evaluated. 

I.  Manpower 

Labor  quantities  and  skills  required  for 
production  are  reviewed  and  determined 


if  adequate.  Worker  efficiency  factors, 
certification  of  workers  in  critical  manu¬ 
facturing  processes,  adequacy  of  skills 
and  employee  development  plan  to  im¬ 
prove  capabilities,  productivity  and 
efficiency  are  all  reviewed. 


CONTROL  OF  CRITICAL  ITEMS 

John  Q.  A.  Wickham 

QUALITY  ENGINEERING  SPEaALIST 
GENERAL  DYNAMICS/CONVAIR 


SUMMARY 

Critical  items  are  weapon  or  space  system 
hardware  assemblies  that  must  operate  with¬ 
out  loss  of  function  in  their lend  usage,  else 
the  system  experiences  catastrophic  failure. 

Critical  items  and  their. control  are  discuss¬ 
ed  in  terms  of  their  Government  specification 
requirements  and  selection.  Experienced 
control  elements  and  techniques  are  detailed. 
Issues  arising  out  of  experience  in  initiating 
and  accomplishing  a  critical  items  program 
are  discussed. 

•  Should  most  time  and  money  for  a  critical 
items  program  be  concentrated  on  sub¬ 
contractor  items? 

•  Is  the  cost  of  control  significantly  less  than 
potential  program  losses  with  significant 
probabilities? 

•  How  is  the  stumbling  block  of  "proprietary 
processes"  best  negotiated? 

SPECIFICATIONS 

Government  specification  of  requirements  and 
control  of  critical  items  is  documented  in 
NHB  5300.4  (1^-2),  "Safety,  ReUability, 
Maintainability  and  Quality  Provisions  for  the 
Space  Shuttle  Program,"  and  MIL-STD-1535A, 
"Supplier  Quality  Assurance  Program  Require¬ 
ments,  MlL-STD-1543,  Reliability  Program 

(WTvr'TTnv  Requirements  for  Space  and 
SELECTJQN  Missile  Systems," 

The  specifications  call  for  selection  of  criti¬ 
cal  items  by  accomplishment  of  failure  modes 
and  effects  analyses  (FMEAs),  which  define 
mission  critical  failure  modes.  Figure  1 
illustrates  an  FMEA  example.  The  specifi¬ 
cations  call  for  special  attention  to  the  speci¬ 
fication,  manufacturing  operations,  inspections, 
and  test  planning  relating  to  selected  critical 
items.  Selection  of  critical  items  is  temper¬ 
ed  by  considerations  of  complexity  and 


program  peculiar  requirements,  such  as  cus¬ 
tomer  desired  e3q)erience  data.  Critical 
items  tend  to  be  hardware  assemblies  that 
are  field  replaceable. 

MIL-STD-1535A,  being  a  supplier  control 
document,  considers  subcontractor  supplied 
critical  items  (registered  components).  The 
program  principal  contractor  might  tend 
towards  selection  of  subcontractor  supplied 
items  since  he  knows  less  about  them  com¬ 
pared  to  his  own  hardware.  The  customer 
normally  has  more  insight  into  the  contractor 
hardware  than  supplied  items.  Subcontractor 
proprietary  processes/configurations  can  be 
a  principle  stumbling  block  to  establishing  a 
critical  item  program.  Clear  communication 
between  the  subcontractor/contractor  is  all 
important  to  minimize  this  potential  problem. 

CONTROL  ELEMENTS  AND  TECHNIQUES 


A  critical  item  control  plan  is  documented 
which  lists  the  critical  items  along  with  their 
data  requirements  and  required  processing/ 
inspection  methods.  These  requirements  and 
melhods  are  reviewed  and  approved  by  the 
contractor. 

The  plan  normally  becomes  a  part  of  the  pro¬ 
gram  quality  assurance  plan,  see  Figure  2. 

It  also  may  be  converted  to  a  data  require¬ 
ment  for  subcontractor  response. 

Crilical  items  and  their  requirements  should 
also  be  documented  in  released  engineering  to 
assure  change  control  and  provide  authority 
for  imposition  of  the  requirements  in  factory 
planning  and  procurement. 

Aerospace  and  military  programs  concerned 
with  relatively  few,  highly  expensive,  high 
reliability  systems  should  consider  the 
pedigreed  critical  item  as  a  method  of  control. 
Such  a  critical  item  carries  its  total,  detail¬ 
ed  success/failure  history  with  it  through 
fabrication  and  test  until  end  use.  Its  history 
is  evaluated  (flight  certified)  by  a  design/ 
reliability  engineering  team  at  predetermined 
milestones  to  determine  end  use  acceptability. 
This  evaluation  is  over  and  above  normal 
inspection  and  is  accomplished  to  detect 
adverse,  in- tolerance  performance  trends  as 
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well  as  possible  operating  overstresses  and 
unresolved  functional  failures.  Figure  3  is  a 
flow  diagram  depicting  the  generation  and 
compilation  of  the  failure/success  data  as 
well  as  the  subsequent  evaluation  (flight 
certification)  by  the  engineering  team. 

The  requirements  for  pedigreed  critical 
items  are  documented  in  a  released  engineer¬ 
ing  document.  These  requirements  are  then 
called  out  in  the  factory  planning  traveler 
which  accompanies  the  item  through  produc¬ 
tion  and  test.  Inspection  initiates  and  main¬ 
tains  a  compilation  of  build,  test  and  rejection 
history  (history  jacket)  in  compliance  with 
the  planning  callouts.  This  history  jacket  is 
reviewed  by  the  Design/Reliability  Engineer¬ 
ing  Team  (Flight  Certification  Board)  upon 
completion  of  acceptance  testing.  Upon  flight 
certification  by  the  Flight  Certification  Board 
(FCB)  and  installation  of  the  critical  item  into 
its  end  system,  the  history  jacket  Is  supplant 
ed  by  a  log  (System  Event  Log)  chronological¬ 
ly  listing  system  events  at  vehicle  level.  The 
System  Event  Log  with  supporting  test  data  is 
flight  certified  at  the  vehicle  launch  site  prior 
to  flight. 

ISSUES  ARISING  FROM  CONTROL  OF 
CRITICAL  ITEMS 

A  critical  item  control  activity  adds  signifi¬ 
cant  cost  to  the  overall  program.  It  is  im¬ 
portant  to  compare  that  cost  to  the  risks  to 
the  overall  program  without  critical  items 
control. 

It  appears  that  it  might  be  program  beneficial 
to  concentrate  critical  item  control  activity 
on  subcontractor  items  rather  than  contractor 
items  since  less  information  is  available  for 
the  former  items. 

Classification  of  processes  by  the  sub¬ 
contractor  as  "proprietary"  represents  a 
potential  stumbling  block  that  requires  clear 
and  definitive  communications  between  the 
subcontractor  and  contractor. 

It  is  vital  that  critical  item  control  plans  be 
tailored  for  each  item.  Boilerplate  plans 
tend  to  exaggerate  the  size  and  cost  of  the  task 
in  the  eyes  of  the  subcontractor. 


Figure  1.  Example  of  failure  modes  and  effects  analysis. 


MISSION  ASSURANCE  PLAN 


SUBJECT:  CENTAUR  0-1  CRITICAL  ITEMS  LIST  (COMPONENTS 


A1 


Date 

1  Oecambtr  197S 


Pace  1  of  6 


Prime  Specification 
Chapter/Scction/Parapaph  No. 


NHB  5300.4(1 A) 


NHB  5300.4  (IB) 


SAMSO-LW-002 


APPENDIX  A-1 

CENTAUR  D.1  CRITICAL  ITEMS  LIST  (COMPONENTS) 


The  items  listed  below  as  requiring  history  jackets 
and/or  data  submittal  are  correct  as  of  1  December 
1975.  This  requirement  is  constantly  changing  and 


•FLIGHT  LEVEL  ACCEPTANCE  TEST 

SYSTEM  023,  ATLAS/CENTAUR  SEPARATION 

55-74355 

Detonation  transfer  assembly 

55-74365 

Dual  detonator  assembly 

55-75882 

Shaped  charge  assembly 

SYSTEM  025,  INSULATION  PANELS  AND 

SEPARATION 

55-74352 

Shaped  charge  assembly 

55-74355 

Detonation  transfer  assembly 

55-74362 

Shaped  charge  assembly 

55-74365 

Dual  detonator  assembly 

SYSTEM  027,  NOSE  FAIRING  AND 

SEPARATION 

27-76275 

Jettison  sys.  actuator  assy,  (fwd] 

55-06018 

Explosive  cartridge 

55-07057 

Explosive-actuated  bolt 

55-07103 

Explosive  cartridge 

55-08019 

Explosive  cartridge 

55-74355 

Detonation  transfer  assembly 

55-74365 

Dual  detonator  assembly 

55-74382 

Shaped  charge  assembly 

will  be  kept  current  only  in  Convair  publication 
55-00091  (AC-CCR). 
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Figure  2.  Example  of  a  Critical  Item  Control  Plan 
as  Part  of  a  Quality  Assurance  Plan 


421 


— NBWAL  OVNAMieg 
C0m»ir  Oi¥i$hm 

CONTROL  OF  CRITICAL  ITEMS 


RELEASED  TOTAL  FLIGHT  VEHICLE  FLIGHT 

ENGINEERING  COMPONENT  CERTIFICATION  LEVEL  CERTIFICATION 

DOCUMENT  HISTORY  IN  FACTORY  AT  LAUNCH 

SITE 


Figure  3.  Controlled  components  -  major  events 


A  NEW  DIMENSION  FOR  QUALITY 

B.  J.  (Bill)  Chumbley 

BELL  HELICOPTER  TEXTRON 

CAD/CAM  is  currently  being  pro¬ 
moted  as  the  most  effective  way  of 
providing  the  ultimate  in  ef¬ 
ficient,  yet  error-free  design, 
manufacturing  and  control  disci¬ 
plines.  By  the  elimination  of  the 
human  element,  computer  controlled 
machines  will  assure  that  the  same 
function  will  be  performed  the 
Scime  way  every  time.  In  theory, 
this  appears  to  be  the  Utopia  that 
we  have  been  searching  for.  In 
practice,  however,  it  has  become 
very  clear  that,  for  the  system  to 
work  effectively,  periodic 
measurements  and  adjustments  must 
be  made. 

A  computer  cannot  act.  It  can 
only  react  to  programmed  stimuli. 
Human  beings  must  program  the  com¬ 
puter  before  it  can  react.  Humans 
are  subject  to  human  error.  Thus, 
the  ability  of  a  computer  con¬ 
trolled  activity  to  function  ac¬ 
curately  depends  upon  the  accuracy 
of  the  programmer.  When  the  ad¬ 
ditional  factors  of  machine  re¬ 
peatability,  tool  wear  and  proper 
placement  of  tools  and  fixtures 
are  introduced,  the  probability  of 
an  error-free  operation  begins  to 
diminish.  Left  uncontrolled,  the 
process  can  rapidly  deteriorate 
into  a  state  of  unprofitability. 

Although  the  traditi9nal  methods 
employed  to  design  and  produce  a 
product  have  been  altered  by  the 
use  of  the  computer,  the  basic 
goal  of  providing  the  customer 
with  a  product  that  meets  his  ex¬ 
pectations  remains  intact.  The 
areas  of  quality  assurance,  how¬ 
ever,  must  now  be  shifted  up¬ 
stream  into  the  manufacturing 
cycle  and  ultimately  into  the  de¬ 
sign  phase  if  the  CAD/CAM  system 
is  to  be  economically  controlled. 
This  then  requires  additional 
emphasis  being  placed  on  the  soft¬ 
ware  portion  of  the  package  and 
modified  prograims  of  process 


control  in  lieu  of  only  inspecting 
the  finished  part. 

Because  of  the  relative  "newness" 
of  CAD/CAM,  program  definitions 
used  to  control  this  concept  re¬ 
main  rather  obscure  and  vary  from 
one  company  to  another.  Basically, 
this  problem  of  uniform  control 
principles  can  be  cited  in  three 
"lacks": 

1.  The  lack  of  understanding  of 
how  it  works. 

2.  The  lack  of  a  set  of  com¬ 
monly  accepted  controls. 

3.  The  lack  of  a  uniform  docu¬ 
mentation  requirement. 

In  this  paper,  we  will  examine 
what  CAD/CAM  consists  of  so  that 
the  other  two  areas  can  be  better 
addressed.  In  the  first  part,  we 
will  provide  answers.  In  the 
other  part  we  will  provide  only 
facts  upon  which  decisions  must  be 
made  in  order  to  provide  an  ef¬ 
fective  means  of  control. 

To  begin,  let's  define  what  CAD/ 
CAM  actually  is.  CAD/ CAM  is  the 
acronym  that  has  been  coined  by 
our  computer  people  to  describe  a 
philosophy  of  creating  a  totally 
computerized  automated  system  for 
designing  and  producing  a  product. 
It  is  not  an  art.  Neither  is  it 
an  exact  science.  The  CAD  portion 
stands  for  Computer  Aided  Design. 
The  CAM  portion  stands  for  Com¬ 
puter  Aided  Manufacturing.  The 
slash  between  the  CAD  and  the  CAM 
is  placed  there  to  indicate  an 
"either  or"  situation.  Either  one 
of  the  systems  can  be  used  sepa¬ 
rately  or  in  conjunction  with  one 
another . 

The  CAD,  or  Computer  Aided  Design, 
deals  with  the  design  portion  of 
the  system  and  involves  the  use  of 
computer  graphics  to  assist  in  the 
calculations  and  physically  de¬ 
fining  the  part  to  be  designed  and 
programmed.  In  the  place  of  the 
traditional  drafting  table,  calcu¬ 
lator,  slide  rule,  reams  of  paper 
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and  numerous  volumes  of  reference 
data,  the  designer  is  equipped 
with  a  console  (see  fig.  1) .  The 
tools  of  his  trade  consists  of  a 
picture  tube  (or  CRT) ,  a  type¬ 
writer  keyboard,  a  function  se¬ 
lector  box  and  a  light  pen.  When 
the  screen  is  turned  on,  there 
will  appear  a  series  of  function 
listings  above  and  below  the 
borders  shown  on  the  CRT.  The  de¬ 
signer  may  select  any  of  the  pro¬ 
grammed  functions  by  merely  touch¬ 
ing  the  screen  at  the  position  he 
wishes  the  function  to  appear,  and 
then  punching  the  proper  button  on 
the  function  selector  box.  An 
image  of  this  maneuver  will  ap¬ 
pear  on  the  screen.  The  designer 
continues  to  repeat  these  ma¬ 
neuvers  until  the  design  is  com¬ 
plete.  He  then  punches  another 
button  and  the  program  is  trans¬ 
mitted  to  the  computer  for  storage. 
If  hard  copies  of  the  design  are 
needed,  provisions  can  be  made  to 
have  the  computer  print  either  an 
aperture  card  or  paper  print. 

Manufacturing  planning  activities 
also  enjoy  the  benefits  of  in¬ 
creased  accuracy  and  time  savings 
by  the  use  of  the  CAD  system.  The 
planner  may  call  the  design  up  on 
the  screen  and  progressively  plan 
the  manufacturing  steps  in  reverse 
order  of  fabrication.  (Thus,  he 
starts  with  a  finished  part  and 
plans  back  to  the  starting  ma¬ 
terial  configuration.)  As  the  ma¬ 
chine  sequences  are  being  formu¬ 
lated,  NC  tapes  (or  programs), 
holding  fixtures  and  cutter  con¬ 
figurations  can  also  be  designed 
and  ordered.  When  the  planner  is 
finished,  his  input  is  also 
stored  within  the  computer. 

Now,  let's  explore  the  CAM  portion 
of  CAD/CAM.  This  is  short  for 
Computer  Aided  Manufacturing  and 
principally  applies  to  the  con¬ 
trol  of  metal  cutting  machines. 

A  typical  basic  machine  tool  is 
equipped  for  three  directions  of 
movement,  as  illustrated  in  fig.  2. 
The  work  platform  can  be  moved 
from  side  to  side  or  in  and  out. 


The  head,  or  cutting  tool  holder, 
is  equipped  with  up  and  down  move¬ 
ment  capabilities.  On  some  models 
the  up  and  down  movement  is  pro¬ 
vided  by  moving  the  work  platform 
up  or  down,  while  the  cutting  tool 
holder  remains  stationary.  If  we 
label  these  three  axes  of  travel 
X,  y  and  z,  and  assign  numerical 
values  along  each  axis ,  we  can 
then  pinpoint  the  exact  location 
and  depth  of  the  cutting  tool  upon 
an  imaginary  grid  on  the  machine. 

If  we  choose  to  attach  a  servo 
motor  to  each  of  the  lead  screws 
that  propel  the  work  platform 
along  the  x  and  y  axes  and  the 
head  along  the  z  axis,  we  can 
write  a  program  that  directs  the 
machine  to  relocate  to  a  specific 
position  by  specifying  the  x,  y, 
and  z  location  in  a  numerical 
equation.  Thus,  the  term  "numeri¬ 
cal  control"  (or  NC)  was  coined  to 
denoce  this  family  of  machines. 

There  have  been  four  generations 
of  NC  machines  developed  within 
the  past  30  years.  These  are 
known  as  NC,  DNC,  CNC,  and  DPNC. 

NC,  simply  called  Numerical  Con¬ 
trol,  consists  of  a  machine  tool 
capable  of  receiving  its  in¬ 
structions  from  an  integrated  con¬ 
troller  on  the  machine.  The  con¬ 
troller  reads  the  program  from  a 
punched  tape,  processes  the  lan¬ 
guage  into  electrical  impulses 
which,  in  turn,  operates  the  ma¬ 
chine.  This  family  of  machines  is 
commonly  referred  to  as  "tape"  ma¬ 
chines  . 

The  first  attempts  at  a  direct 
hookup  between  a  machine  tool  and 
a  computer  produced  a  generation 
of  machine  tools  known  as  DNC. 

DNC  consists  of  a  computer  in 
which  a  machine  program  has  been 
stored;  a  controller  that  is 
capable  of  receiving  the  program 
from  the  computer  and  translating 
it  into  machine  commands;  and  ma¬ 
chines  equipped  to  receive  the 
commands  from  a  controller.  In 
this  set-up,  one,  two,  or  any 
multiple  of  machine  tools  may  be 


hooked  into  this  system  and  oper¬ 
ated  by  a  single  controller.  Two 
drawbacks  exist  with  this  system, 
however.  First,  all  machines  can 
produce  only  one  part  configu¬ 
ration,  and,  if  the  computer  or 
controller  breaks  down,  the  whole 
system  goes  down.  The  major  ad¬ 
vantage  of  a  DNC  system  is  the 
central  control  of  a  battery  of 
machines  with  one  piece  of  program 
media . 

As  the  computer  technology  ad¬ 
vanced,  computers  became  smaller 
and  less  espensive.  Be  equipping 
a  microprocessor  with  a  memory  and 
input/output  circuitry,  a  micro¬ 
computer  small  enough  to  be  in¬ 
cluded  on  an  NC  is  obtained. 
Presently,  this  system  of  control 
is  the  most  predominant  within  the 
industry.  Known  as  CNC,  the  ma¬ 
chine  receives  its  coordinate  com¬ 
mands  from  a  program  stored  in  its 
integral  computer.  Permanent 
storage  of  the  machine  programs 
is  provided  in  a  main,  or  parent, 
computer  and  are  fed  to  the  ma¬ 
chine  upon  demand. 

The  fourth  generation  of  NC  ma¬ 
chine  tools  are  beginning  to  make 
an  appearance  on  the  scene.  Ma¬ 
chine  tools  within  this  category 
are  equipped  with  the  capabilities 
of  both  receiving  and  transmit¬ 
ting  programs  to  the  storage  (or 
host)  computer.  In  addition  to 
reacting  to  program  directions  as 
a  CNC  does,  programs  can  be  modi¬ 
fied  on  the  machine  and  the  modi¬ 
fied  program  transmitted  back  to 
the  host  computer.  , Additional  at¬ 
tachments  may  be  added  to  this  ma¬ 
chine  tool  that  will  allow  punched 
tapes  or  computer  programs  to  be 
made  while  a  part  is  being  ma¬ 
chined.  Upon  completion,  these 
programs  may  be  transmitted  to  the 
host  computer  for  permanent 
storage.  Because  of  the  two-way 
communication  channel,  this  ma¬ 
chine  tool  concept  is  known  as 
Distributive  Process  Numerical  Con¬ 
trol  ,  or  DPNC . 

From  the  above  discussion,  it 
would  appear  that  computer  tech¬ 


nology  has  provided  almost  all  of 
the  answers  for  an  efficient  manu¬ 
facturing  system.  One  factor, 
however,  remains  unresolved.  This 
is  the  question  of  economical,  yet 
efficient  product  assurance  at  the 
customer  level.  Consumer  demands 
for  product  safety  and  reliability 
considerations  have  been  augmented 
by  government  regulations  and  con¬ 
sumer  advocate  activities.  This 
results  in  added  pressure  on  the 
producer  (and  especially  his 
Quality  Department)  to  take  every 
reasonable  precaution  of  assuring 
that  only  good  units  of  product 
are  offered  for  sale. 

Inspection  (i.e.,  physical 
measurement  of  a  finished  part)  is 
the  traditional  method  of  assuring 
that  the  parts  passed  on  are  with¬ 
in  the  limits  of  the  design  speci¬ 
fications.  Before  the  intro¬ 
duction  of  NC,  one  inspector  could 
normally  inspect  the  work  of  eight 
machinists  with  a  reasonable  de¬ 
gree  of  accuracy.  Today,  one  NC 
machine  can  perfox'm  the  work  of 
five  to  twelve  conventional  ma¬ 
chines,  while  holding  intricate 
dimensional  relationships  and 
tolerance  never  before  attainable. 
Parts  can  be  produced  in  a  few 
hours  that  will  require  several 
hundred  hours  to  measure  when  tra¬ 
ditional  layout  methods  are  used. 
Of  paramount  importance,  however, 
is  the  fact  that  inspection  only 
sorts  the  good  from  the  bad.  It 
does  not  prevent  the  defect  from 
occurring . 

Quality  Control,  in  its  ultimate 
form,  assures  that  the  product 
meets  all  previously  established 
requirements  and  that  the  process 
is  controlled  within  specified 
limits.  Again,  however,  the  ef¬ 
fectiveness  of  a  quality  control 
system  depends  upon  its  ability  to 
measure  the  product  in  a  timely 
manner . 

American  Machinist  Magazine  de¬ 
scribes  the  science  of  Quality  As¬ 
surance  as  a  predictive  process. 
"To  be  effective,  it  requires  as¬ 
sessment,  monitoring,  and  control. 


Every  process  in  the  manufacturing 
cycle  of  a  product  must  first  be 
as^"  3ssed  to  determine  its  po- 
texitial  for  meeting  previously  es¬ 
tablished  quality  requirements. 
Once  the  process  is  selected,  it 
must  be  monitored  to  make  sure 
that  the  assessment  of  its  po¬ 
tential  is  still  correct.  Finally, 
the  process  must  be  controlled  on 
the  basis  of  such  monitoring  to 
keep  the  process  under  control" 
Under  the  terms  of  this  treatise, 
every  process  in  the  manufacturing 
cycle  must  be  assessed.  This 
means  that  Quality  must  gain  the 
expertise  in  the  computer  tech¬ 
nology-related  manufacturing 
practices,  as  well  as  con¬ 
ventional  machining  modes,  in 
order  to  properly  assess  the  manu¬ 
facturing  process  currently  in  ef¬ 
fect  in  the  present  hybrid  ma¬ 
chine  mix  employed  by  most  shops. 
As  more  and  more  computer  tech¬ 
nology  is  introduced,  this  re¬ 
quirement  for  computer-oriented 
Quality  Engineers  will  become 
increasingly  acute. 

In  the  last  decade,  the  emphasis 
of  computer  technology  has  been 
applied  to  producing  the  part. 
Little  effort  has  been  made 
across  the  industry  to  provide 
equipment  that  will  economically 
measure  the  product  during  se¬ 
lected  stages  of  manufacture.  As 
a  result,  each  company  has  at¬ 
tempted  to  design  their  own  in¬ 
spection  screens.  This  has  often 
been  accomplished  by  selective 
sampling,  composite  gaging,  and 
reliance  on  the  NC  machine  to 
produce  a  good  part.  Partially 
because  of  the  lack  of  computer 
technology  expertise  and  princi¬ 
pally  due  to  the  lack  of  adequate 
funding,  the  Quality  discipline 
has  lagged  behind  the  accelerated 
advancement  of  its  production 
counterpart. 

Quality  Assurance,  as  a  disci¬ 
pline,  must  re-examine  its  po¬ 
sition  of  Product  Assurance,  es¬ 
pecially  in  the  area  of  machined 
parts.  In  high  volume,  continu¬ 
ous  production  environments,  the 


use  of  sample  inspections  and  con¬ 
trol  chart  concepts  have  proven  to 
be  adequate  for  controlling  the 
quality  of  the  end  product. 

Process  controls  have  been  demon¬ 
strated  to  be  successful  in  the 
control  of  heat  treatment  and 
chemical  process  operations. 

First  part  inspections,  combined 
with  sampling  of  the  finished  lot 
is  also  a  popular  means  of  con¬ 
trol.  Although  these  methods  have 
been  proven  to  be  successful  in 
the  past,  their  effectiveness  in 
controlling  the  machining  concepts 
of  tomorrow  will,  at  best,  be 
marginal.  Consider  for  a  moment 
these  facts: 

o  Lot  sizes  are  now  less  than 
50  pieces  for  75%  of  all  pro¬ 
duction  runs. 

o  90%  of  all  NC  machine  pro¬ 
grams  take  less  than  27 
minutes  to  run.  48%  are  run 
in  less  than  10  minutes. 

o  Rework  of  part  nonconformi¬ 
ties  are  almost  non-existent 
on  most  NC  machines. 

Once  these  facts  are  examined,  in 
the  light  of  today's  Product  As¬ 
surance  programs,  it  is  immediate¬ 
ly  apparent  that  additional  exper¬ 
tise  and  measurement  capabilities 
must  be  obtained.  Otherwise,  pro¬ 
hibitive  scrap  and  rework  costs 
and  excessive  inspection  bottle¬ 
necks  will  impede  the  progress  of 
improved  machining  productivity. 
The  other  alternative  of  relying 
on  the  machine's  ability  to  pro¬ 
duce  only  good  parts  could  prove 
disasterous  in  light  of  today's 
product  liability  litigations. 

A  successful  quality  assurance 
program  for  the  CAD/CAM  concept 
must  begin  at  the  earliest  stage 
of  the  program — the  design  phase. 
Decisions  must  be  made  to  how  and 
where  the  product  will  be  measured 
for  compliance.  Inaccessible  di¬ 
mensional  relationships  and  speci¬ 
fication  contraditions  should  be 
resolved  before  the  manufacturing 
effort  is  initiated.  Additional 


measurement  equipment  and  programs 
that  will  be  required  to  verify 
the  product  can  be  ordered  in  ad¬ 
vance  to  assure  its  availability, 
when  needed.  In  many  companies, 
modified  forms  of  this  program  are 
currently  in  effect.  Most  of  the 
evaluations,  however,  are  cur¬ 
rently  made  from  a  formal  blue¬ 
print,  after  the  design  has  been 
finalized. 

In  the  CAD/CAM  era,  two  new 
factors  must  be  considered.  The 
blueprint  will  be  a  video  por¬ 
trayal  on  a  scope  and  the  ma¬ 
chining  will  be  performed  in  a 
fraction  of  the  conventional  ma¬ 
chining  time  on  a  computer  con¬ 
trolled  machine.  In  order  to  per¬ 
form  an  adequate  assessment  of  the 
design  and  develop  an  effective, 
yet  economical  verification  pro¬ 
gram,  the  Quality  man  must  have 
sufficient  expertise  in  both 
computer  technology  and  shop 
practices . 

Since  operator  compensation  can¬ 
not  be  made  for  such  variables  as 
worn  lead  screws,  slack  bearings 
and  cutter  wear  on  a  NC  machine, 
maintenance  and  calibration  con¬ 
trols  are  of  paramount  importance. 
From  experience,  we  have  proven 
that  the  desired  metallurgical 
properties  can  be  obtained  during 
heat  treatments  if  we  adequately 
control  the  furnace  and  sample 
the  lot  of  material.  These  same 
results  can  reasonably  be  ex¬ 
pected  from  an  NC  machine  if  ade¬ 
quate  evaluations  are  scheduled 
in  a  timely  manner.  Several 
factors  must  be  considered  in  de¬ 
signing  such  a  progreim.  Among 
the  more  prevalent  areas  are  the 
following: 

o  Machine  Installation:  The 
initial  machine  installation , 
leveling  and  adjustments  made 
should  be  verified.  The  use 
of  a  laser  interferometer 
for  verification  of  the  in¬ 
stallation  can  provide  the 
required  degree  of  accuracy 
at  a  reasonable  cost. 


o  Periodic  Calibration:  Peri- 
odic  calibration  checks  on  a 
machine  will  assure  that  the 
machine  is  still  capable  of 
maintaining  the  tolerance  al¬ 
lowables  of  a  design.  The 
use  of  test  programs  to  make 
a  test  part  with  given  di¬ 
mensions  approximating  all  of 
the  machine  moves  have  been 
used  quite  successfully.  Po¬ 
sition  checks  measured  by 
laser  interferometer  have 
also  proven  adequate . 

o  Tooling  and  Cutter  Control: 
Quality  verification  of  tool 
holder  settings,  tool  regrind 
and  holding  fixtures  will 
provide  an  additional  degree 
of  assurance.  Cutter  wear 
studies  provide  for  changing 
the  cutting  tool  before  a 
nonconformity  appears. 

Automated  inspection  equipment  is 
beginning  to  appear  on  the  market. 
Computerized  coordinate  measuring 
systems  are  now  available  that  can 
provide  a  wealth  of  analyzed  data. 
Non-contact  gaging  concepts  are 
being  pursued  by  many  companies, 
among  the  more  notable  being 
United  Technologies  Research  Cen¬ 
ter  (laser  diodes) ,  EMI  Photo- 
Electric  (Comp  Gage  System) ,  and 
Jones  and  Lamson  (Metric  Eye) . 
Total  system  quality  control  is 
the  goal  of  Lockheed  Missile  & 
Space  Company's  IPQC  System.  The 
ultimate  selection  of  the  equip¬ 
ment  best  suited  for  each  company 
must  be  made  by  the  Quality  De¬ 
partment.  Since  each  piece  of 
this  equipment  is  expensive  and 
unique,  decisions  must  be  based 
on  application,  operator  fa¬ 
miliarization,  maintenance  and 
economic  considerations. 

To  further  compound  the  situation 
are  two  new  innovations :  Auto¬ 
matic  Adaptive  Control  of  Machine 
Tools  and  Group  Technology.  Under 
the  Adaptive  Control  concept,  the 
workpiece  will  be  continually 
monitored  and  the  necessary  ad¬ 
justments  to  maintain  the  required 
accuracy  at  optimum  machining 


rates  are  automatically  made. 

Group  technology  consists  of 
grouping  families  of  similar  parts 
so  that  they  can  be  produced  more 
efficiently.  Quality  Assurance 
inputs  during  the  definition  and 
design  stages  of  both  of  these 
programs  are  necessary  to  insure 
a  successful  implementation. 

The  building  blocks  for  tomorrow's 
quality  programs  must  begin  to  be 
carefully  selected  today  if  we  are 
to  be  successful  tomorrow.  This 
must  begin  with  personnel,  not 
only  fully  knowledgeable  of 
today's  requirements,  but  also 
with  sufficient  knowledge  of  com¬ 
puter  technology-related  systems 
currently  under  development  to 
maintain  pace  with  manufacturing 
advancements.  Old  concepts  must 
be  re-evaluated  and  adjustments 
be  made  accordingly. 

The  computer  has  provided  us  a 
tool  with  which  we  can  produce 
goods  and  services  at  a  fraction 
of  the  time  previously  required. 

To  assure  that  these  goods  and 
services  are  continually  produced 
in  a  prescribed  manner  requires  a 
system  of  controls  and  quality  as¬ 
sessments.  In  order  to  design  and 
maintain  this  system  of  controls 
and  quality  assessments,  a 
knowledge  of  the  computer  tech¬ 
nology-related  manufacturing 
principles  must  be  present. 


Figure  2 

Basic  Machine  Movements 
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THE  CRAFTSMANSHIP  WORKSHOP 
SUMMARY  OF 

ISSUES  AND  RECOMMENDATIONS 


1.  Craftsmanship  Shortage 
Issue 

Lack  of  skilled  craftsmen 

-  Aerospace 

-  Nationally 

Industry's  blue  collar  work  force 
is  aging.  New  highly  qualified 
people  must  be  developed  to  re¬ 
place  them. 

Recommendations 

oFor  critical  crafts 

-  Apprentice  training-contract 
line  item. 

-  Proposed  legislation  required. 

oA  broad  training  program  must 
be  established  for  crafts  and 
trades. 

oMore  extensive  use  must  be  made 
of  the  "pink  collar"  work  force. 

o Skill  qualification  training  must 
be  stressed. 

o  Job  proficienty  demonstration 
must  be  ongoing. 

2.  Quality  in  Source  Selection 
Issue 

Raise  emphasis  on  quality  in 
government  source  selection  pro¬ 
cess. 

Recommendations 

oPast  quality  performance  of  a 
contractor  should  be  given  more 
weight  during  source  selection. 

o Specific  criteria  should  be 
emphasized  more  in  the  evalua¬ 
tion  of  contractor's  past  quality 
performance. 


3.  SOW  Clarity 
Issue 

Statement  of  work  (SOW)  clarity 
on  specifications. 

-  cost  driver 

Recommendations 

oSOW's  should  clearly  specify: 

-  mandatory  compliance  vs 
reference  specifications 

-  applicability  of  subtier 
specifications 

4.  Computer  Format 
Issue 

The  aerospace  industry  does  not 
have  a  common  computer  format 
for  CAD/CAM/CAT. 

Recommendation 

©Industry  and  government  estab¬ 
lish  a  common  computer  format 
for  CAD/CAM/CAT, 

-  need  industry/government 
committee 

5.  PRR's 
Issue 

Regulation  84-2  on  production 
readiness  review  should  be  up¬ 
dated.  Production  readiness  re¬ 
view  (PRR)  needs  updating  to 
accomodate  low  volume  programs: 

-  presently  related  to  dollar 
value 

-  tailoring  option  not  being  fully 
implemented. 

Recommendations 

o Update  language  for  clarity  and  in¬ 
corporation  of  latest  experience. 

o Update  structure/format  to  allow 
for  tailoring. 

o  Emphasize  tailoring  PRR  for 
small  programs. 


o  PRR  should  be  addressed  in 
RFP. 

6.  Workmanship  Standards 
Issue 

Government  workmanship  standard 
being  developed  without  apparent 
inter-departmental  and  agency 
inputs. 

Expediency  lacking  for  state-of- 
the-art  changes. 

Recommendations 

o  Expand  committee  membership 

-  design  (packaging)  and  manu¬ 
facturing  engineering  inputs 

-  NASA,  Navy  participation  ■ 

o  Committee  to  address  change 
and  sustaining  process. 

7.  Quality  Planning 
Issue 

Lack  of  quality  planning  and  eval¬ 
uation  during  development  re¬ 
sults  in  unanticipated  problems 
showing  up  in  later  contract  phases, 
pha  s  e  s . 

Recommendations 

o  Develop  a  requirements  document 
for  the  development  phase  stress¬ 
ing  quality  planning  function. 

8.  MIL-STD-  1535A 
Issue 

MIL-STD-  1  535A  is  an  unnecessary 
cost  driver  based  on  current 
experience. 

Basic  requirements  covered  in 
other  specifications. 

Recommendations 

o  Initiate  action  to  cancel  MIL-STD- 
1535A  for  future  contracts. 


9.  Corrective  Action 
Issue 

All  agencies  do  not  use  the  same 
problem  reporting/corrective 
action  system. 

Recommendation 

o Shuttle  Program 

-  develop  uniform  problem 
reporting/cc  rrective  action 
system. 

10.  Conflicting  Specs. 

Is  sue 

Obsolete  and  conflicting  specifi¬ 
cations  exist  due  to  state-of-the- 
art  changes  (i.  e.  ,  MIL-H-6061, 
ANA  616,  and  MIL-P-5510C  vs 
MSEC  SOM  60240). 

-  potential  technical  and  cost 
impact 

Recommendations 

o  Convene  joint  government/indust¬ 
ry  committee  to  review  and  scope 
problem. 

11.  Duplicate  Audits 
Issue 

Excessive  and  duplicate  contractor 
audits. 

Recommendations 

o  Establish  procedures  for  govern¬ 
ment  wide  participation  and 
acceptance  of  inter-agency  audits 
(AF,  NASA,  Navy,  etc.  ) 

o  Tailor  audits  to  include  positive 
as  well  as  the  negative. 

o  Program  office  should  participate 
in  planning  of  audits. 
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AFSC  Incentive  Policy  and 
Space  Division's  Incentives 


Mr.  Harold  E.  Sharp 
Director  of  Pricing 
Space  Division 


INTRODUCTION 

Lt.  Col.  Dalyrmple 
Space  Division  USAF 


APRIL  1978 

MISSION  assurance  CONFERENCE 


FINDING 

I  HIW  INTEREST  RATES  AND  INFLATION 
DISCOUNT  THE  VALUE  OF  LONG  TERM 
INCENTIVES 


I  CONTRACTORS  SHOULD  BE  INVOLVED  IN 
DEVELOPMENT  OF  INCENTIVE  PLANS 


(  GOVERNMENT/ INDUSTRY  SHOULD  JOINTLY 
EXPLORE  THE  SPACE  SHUTTLE  ENVIRONMENT 
FOR  NEW  INCENTIVE  APPROACHES  AND 
CONSIDERATIONS 


ACTION 

I  SD  IS  STUDYING/IMPLEMENTING: 

-  DISCOUNTING  (PRESENT  VALUE)  OF 

INCENTIVES 

-  ADVANCE  INCENTIVE  PAYMENTS 

-  EARLIER,  URGER  AWARD  FEES 

I  INCREASING  USE  OF  DRAFT  RFP'S 

-  ENCOURAGE  COMMENTS  ON 

PERFORMANCE  INCENTIVES, 
AWARD  FEE  FACTORS 

•  OF  CONTINUING  CONCERN/STUDY 


I  MUST  AVOID  COUNTER-PRODUCTIVE  INCENTIVE  •  SPECIAL  ITEM  DURING  CONTRACT  REVIEW 
STRUCTURE 


t  INCENTIVE  PLANS  SHOULD  BE  WELL 
UNDERSTOOD  BY  ALL  PARTIES 


SPECIAL  ITEM  DURING  CONTRACT  REVIEW 
-  INCREASED  USE  OF  POST-AWARD 
CONFERENCE 


•  INCENTIVES  SHOULD  BE  USED  AS  A  I  INCREASING  USE  OF  AWARD  FEE 

TOOL  FOR  ESTABLISHING  RELATIVE 
IMPORTANCE  OF  REQUIREMENTS 


I  AWARD  FEE  IS  USEFUL  AS  A 
"REPORT  CARO"  FOR  CONTRACTOR 


»  SO  IS  INCREASING  USE  OF  AWARD  'EE 
-  SD  HAS  PREPARED  AN 

"AWARD  FEE  CONTRACTING  GUIDE" 
FOR  USE  OF  PROGRAM  O'FICES 


HOW  HAS  THE  CONTRACTING  ENVIRONMENT  CHANGED? 


I  GENERAL  SLAY  INITIATIVES: 

-  MORE  FIXED  PRICE  CONTRACTS 

-  MORE  USE  OF  POSITIVE-NEGATIVE  "BALANCE" 

-  FEWER  SOLE-SOURCE  ACQUISITIONS 

•  SPACE  SHUTTLE: 


-  SCHEDULE/PERFORMANCE  BECOMING  MORE  DEFINED 
RESOURCES  (*,  PEOPLE)  ARE  EVEN  MORE  CRITICALLY  CONSTRAINED 


HIGH  INTEREST  RATES  AND  INFLATION  ARE  CREATING  EVEN  GREATER 
PROBLEMS  WITH  TIME  VALUE  OF  INCENTIVE  FLOW 


INDUSTRY  VIEWPOINT 

Mr.  James  B.  Gordon 
Manager,  Profit  Planning 
TRW,  INC. 

CONTRACTUAL  PERFORMANCE  INCENTIVES 
FROM  THE 

CONTRACTOR'S  PERSPECTIVE 


1972  &  1974  POD  AUDITS  INCENTIVE  &  CPAF 

Clearly,  many  of  these  appllcatlons-or 
misapplications  problems  have  also  been 
documented  and  recognized,  but  not  solved, 
as  Illustrated  by  these  1972  and  1974  DOD 
Audits.  So  it  is  one  thing  to  have  good 
principles  but  quite  another  to  achieve 
those  principles  in  the  real  world... 


PERSPECTIVE  1969 

I'd  like  to  put  the  subject  of  Contractual 
Performance  Incentives  into  their  proper 
context  or  perspective  and  not  treat  them 
in  isolation  from  related  contractual  and 
real  world  economic  aspects. 

1969  DOD  &  NASA  INCENTIVE  GUIDE 

The  subject  of  Incentive  Contracting  has 
been  well  documented  as  to  concept  and 
proper  application,  at  least  In  theory. 
However,  it  is  not  the  guide  books  which 
themselves  create  basic  problems... 

SELECTION  OF  CONTRACT  TYPES 

Instead,  it  is  the  application  in  practice 
of  these  basic  principles  which  holds  the 
key  to  successful  incentive  contracting. 
There  is  some  concern  on  my  part  that  this 
critical  step  has  yet  to  be  successfully 
accomplished. . . 

BASIC  CHARACTERISTICS  OF  FIXED  PRICE 
&  COST  TYPE  CONTRACTS 

The  significant  differences  between  cost 
type  and  fixed  price  type  contracts  can, 
and  usually  do,  create  substantial  finan¬ 
cial  (negative)  repercussions  if  proper 
type  selection  is  not  achieved. . . 

'79  GAO  STUDY  (LCD-79-108) 

For  example,  when  the  contractor  exceeds 
the  boundaries  of  cost  performance  in¬ 
corporated  into  his  (typically  FPI) 
contract,  the  implied  technical  performance 
incentive  structure  is  lost... and  this  has 
often  happened  on  complex  military 
communication  systems. 


PROFIT  '76  GIVETH  &  DOD  TAKETH  AWAY 

An  Historic  review  of  the  Defense  Industry's 
profitability  was  conducted  as  Profit  '76 
by  DOD  with  high  hopes  £or  positive  answers 
and  respectable/ authoritative  data  points 
for  new  policy  development,  particularly 
to  assign  a  role  to  capital  investment  in 
the  profit  negotiation  process. . .this  was 
done,  but  at  a  price. . . 

COST  OF  MONEY  OFFSET 

Unfortunately,  the  net  result  of  the  much 
heralded  Profit  '76  effort  was  to  retain 
the  profit  status  quo,  so  to  speak,  while 
at  the  same  time  claiming  a  remarkable 
breakthrough  in  encouraging  contractors 
to  increase  their  (cost  saving)  capital 
investments.  "Cost  of  Money"  was  recog¬ 
nized  and  accounted  for,  as  was  a  10% 
emphasis  on  capital  in  the  revised  Weighted 
Guidelines.  Raised,  that  is,  to  completely 
offset  (in  aggregate)  the  added  reimburse¬ 
ment  of  Cost  of  Money.  The  idea,  of  course, 
was  much  like  the  carrot  tied  to  the  stick 
tied  to  the  donkey  to  motivate  his 
performance.  That  showed  how  much  DOD 
really  thought  of  defense  contractor's 
financial  astuteness... 

DOD  POLICY  IN  THE  CURRENT  ENVIRONMENT 

Meanwhile,  on  a  policy  level,  DOD  continues 
to  be  concerned  with  saying  the  right 
things  in  these  difficult  times.  They 
repeatedly  recognize  that  low  average 
profit  rates  on  defense  business  is 
detrimental  to  the  public  interest.  They 
know  that  the  best  industrial  capabilities 
will  be  driven  away  from  the  defense  mar¬ 
ket  if  defense  contracts  are  characterized 
by  low  profit  opportunities. 


ENVIRONMENT  GENERALLY  GETTING  MORE  SEVERE 

But  such  hopeful  concern  Is  a  far  cry  from 
meeting  past  problems,  much  less  in 
accomplishing  real  redress  to  the  ever 
deteriorating  situation  which  we  face  at 
the  present  time.., 

INTEREST  RATE  CHART 

Obviously,  a  key  Ingredient  of  today's 
malaise  is  related  to  the  (unrecovered) 
cost  of  financing,  particularly  a  growing 
concern  with  the  DOD's  Increased  use  of 
fixed  price  type  contracts  with  their 
characteristic  risks  and  poor  (relative 
to  cost  type)  payments  cash  flow... 

PROFIT/ SALES  DISTRIBUTION 
BY  CONTRACT  TYPE 

Far  from  being  a  single  profit  value  in 
practice,  realized  profits  to  sales  returns 
are  actually  quite  scattered  in  their 
distribution  in  the  real  world.  This 
chart  from  a  now  dated  LMI  Study  illu¬ 
strates  the  uncertainty  of  achieving  any 
given  profit  rate,  with  increasing 
uncertainty  as  we  move  toward  the  fixed 
price  type  contracting... 

CASH  FLOW  EXAMPLE  FIXED  PRICE 
VS.  COST  TYPE 

Moreover,  what  masquerades  as  "Higher 
Profit  Opportunities"  on  the  fixed  price 
procurements  are  more  than  offset  by  the 
typically  poor  cash  flow  results  inherent 
with  the  payments  practices  vis  a  vis  cost 
type  payments.  Even  with  milestone  pro¬ 
visions  on  long  term  large  programs,  only 
a  part  of  this  basic  inequity  is  restored 
to  the  contractor,  and  essentially  little, 
if  any,  (net)  improvement  results  from 
milestones  where  the  "price"  of  approval 
equals  the  amount  of  the  inequity  in  the 
f irst  place. , . 

CASH  FLOW  RATES  OF  RETURN 

In  the  real  world,  on  an  after  tax,  after 
capital  Investment  cash  flow  basis  (no 
overrun) ,  an  8%  cost  type  contract  is 
Incrementally  better  than  an  Identical 
p'.oject  at  15%  on  a  fixed  price  basis. 

The  resolution  of  this  financial  flip- 
flop  is  definitely  not  to  restructure  the 
cost  type  payments  to  be  as  bad  as  the 


fixed  price  contracts,  but  to  come  tii 
grips  with  ways  to  alleviate  this  obvi^ul^i 
contradiction.  Thus,  the  "paper"  proi its 
on  fixed  price  contracts  can  be  large] v 
illusory  in  many  cases,  without  a  dime 
of  overrun! 

TREND  SINCE  1976  OF  FIXED  CAPITAL 
INVESTMENT  TO  SALES 

Since  many  corporations  have  DOD  busiiu  . 
as  some  (usually  minor)  fraction  of  tlu  ii 
total  sales,  and  since  they  typically 
measure  financial  performance  in  relai  ion 
to  assets  employed  or  used  in  produc  iny, 
their  resulting  profits,  the  recent 
dramatic  increase  in  capital  assets 
needed  to  perform  DOD  work  has  created  a 
challenge  to  retain,  much  less  improve 
upon  past  profitability  performance  in 
relation  to  assets,  both  fixed  and  casli 
flow  variable  assets  tied  up  under 
current  fixed  price  contracting  payments 
rules. 

RATE  OF  RETURN  ON  ASSETS-SCHEMATIC 

Looking  at  the  basic  elements  of  Return 
on  Assets,  it  is  clear  that  there  are 
limited  options/outcomes  under  circum¬ 
stances  of  rising  capital  employed  (fixid 
&  variable)  with  a  profit... 

RATE  OF  RETURN  FORMULA 

Policy  that  mandates  a  constant  rate  of 
"negotiated"  or  going-in  "markup"  on 
sales.  In  order  to  meet  corporate 
financial  objectives  which  stress... 

ROAE-OPTIONS 

Improved  Return  on  Assets,  you  have  to 
either  reduce  assets  or  increase  sales/ 
profits  enough  to  keep  in  their  good 
graces,  so  to  speak.  But  capital  invisi- 
ment  and  absorption  is  dramatically 
increasing  today  as  a  result  of  inflaLii>n 
and  technology  coupled  with  more  fixed 
fixed  price  contracts  with  accompany  in.', 
lousy  payments,  and  I  would  expect  manv 
contractors  are  waking  up  to  this. 

NET  PROFIT  CONTRACT  EXAMPLE 

While  the  government  typically  recogiii.  i 
only  the  "big  number"  or  "negotiated"/ 
contractual  profit  percentage,  the  cuti- 
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tractor  can  keep  only  a  tiny  fraction  of 
this  figure  in  reality,  and  it  is  the 
latter  figure  on  which  Investment  decisions 
are  made,  not  the  former;  even  without  the 
aspect  of  "overruns"  there  is  a  major  dis¬ 
tinction  to  be  made  between  the  government's 
perception  of  profit  from  that  which  the 
contractor's  banker  sees... 

OVERRUNS  NOT  LIMITED  TO  POD 

Obviously,  DOD  is  not  the  only  agency  or 
entity  to  have  cost  overruns,  as  witnessed 
by  this  chart.  Many  other  procurements 
are  also  afflicted  with  "unknowns"  as  well, 
and  yet  this  remains  today-within  DOD-  a 
real  concern  and  an  unresolved  problems  as 
far  as  its  effect  on  realized  earnings  is 
concerned. 

UNALLOWABLE  COSTS- INCREASING 

Many  other  elements  contribute  to  the 
basic  difference  between  "markup"  profits 
and  real  net  returns.  These  costs  not 
recognized/or  reimbursed  under  government 
contracting  (chart)  are  nonethele  s  real 
and  significant  to  a  contractor,  and  cannot 
be  overldoked  in  any  enlightened  discussion 
of  decision  profitability  or  true  con¬ 
tractual  financial  performance. 

*76/77  DOD  NEGOTIATED  PROFITS 

Therefore,  so-called  negotiated  profits 
(as  used  by  DOD)  have  very  little  signi¬ 
ficance  intrinsically  until  they  are 
properly  placed  within  their  contractual 
context  of  unallowables,  taxes,  investments, 
cash  flow,  etc. 

FINANCIAL  ANALYSIS  METHODS 

As  shown  by  this  summary  of  Financial 
Analysis  methods  from  Investment  '76 
(part  of  Profit  *76),  DOD's  profit  method 
is  last  (and  least)  on  the  list,  with 
practically  no  takers  (1).  The  most  viable 
and  generally  used  approach  (on  major 
programs)  is  a  net  after  tax  cash  flow, 
time-factored  Integrated  analysis. 

GENERAL  SLAY'S  STATEMENT  TO  NOMA 
RE  INCENTIVE  CONTRACTS 

We  are  told  that  AFSC  intends  to  favor 
"balanced"  Incentive  contracts.  This 
translates  to  mean  that  the  contractor 


can  lose  money  for  poor  performance  (but 
no  overrun)  and  win  big  (also  if  no  over¬ 
run)  if  performance  is  perfect. 

GENERAL  SLAY'S  YARDSTICK  OF 
INCENTIVE  QUALITY 

Good  results  in  incentive  contracts 
apparently  focus  on  how  low  you  go  on 
FPI  Celllnes  and  how  high  vou  get  on  share 
ratios.  I  would  hope  that  such  a  state¬ 
ment  also  presumes  a  caveat  that  the 
balance  of  the  contractual  structure  is 
within  achievable  limits  so  that  whatever 
incentives  there  are  can  in  fact  operate 
in  practice  as  well  as  on  a  briefing 
chart. 

LMI  INCENTIVE  CONTRACTING  STUDY 

In  its'  study  of  Incentive  Contracting 
results  (May  1968)  LMI  found  no  signi¬ 
ficant  correlation. . . 

FINDINGS  OF  INCENTIVE  CONTRACTING  STUDIES 

Between  cost  sharing  ratios  and  overruns 
or  underruns.  They  did  find  that  incen¬ 
tives  have  not  been  significantly  effective 
as  protection  against  cost  growth  on 
programs — as  the  latest  1979  GAO  Report 
also  confirmed — and  that  contractors  will 
not  sacrifice  performance  attainment  for 
profit. 

STANFORD  STUDY  (HEMMES) 

In  this  1969  study,  Robert  Hemmes  notes 
in  his  thesis  on  Economic  Motivation  of 
Contractual  Techniques  that  the  results 
demonstrate  that  the  incentive  procurement 
techniques  have  not  improved  procurement 
efficiency  and  that  in  some  cases  strong 
arguments  emerge  that  these  techniques 
have  been  dysfunctional. 

STANFORD  STUDY  (BELDEN) 

In  his  Defense  Procurement  Outcomes  Study 
(also  1969) ,  David  Belden  concludes  that 
defense  procurement  outcomes  in  the  in¬ 
centive  contracting  environment  have  not 
reflected  well  on  the  use  of  incentive 
contracts.  His  primary  recommendation  is 
to  consider  and  reflect  in  the  pricing 
arrangement/contract  type  selection  the 
uncertainties  involved  in  the  procurement 
to  avoid  mismatches. 
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BELDEN  STUDY-FINIS 


COMPUTER  RUN-TEN  PROGRAM  CASE  RESULTS 


Belden  quotes  an  appropriate  observation 
which  is  just  as  valid  today  in  placing 
incentive  contracting  into  its  proper 
perspective  when  he  says  that  DOD's  mission 
is  primarily  or  firstly  to  defend  well  and 
that  to  buy  well  comes  second.  The  fact 
is  that  the  key  problem  of  proper  contract 
type  selection  has  not  been  nor  cannot 
fully  be  resolved  which  inherently  pre¬ 
cludes  perfection  in  incentive  contracting 
practice. 

TRW'S  INCENTIVE  CONTRACTING  SIMULATION 

Even  if  we  assume  away  all  the  real  world 
problems  of  proper  contract  type  matching, 
a  look  at  the  real  world  constraints  via 
our  incentive  contracting  simulation 
program  reveals  another  set  of  practical 
obstacles,  particularly  in  applying  the 
theory  of  orbital  incentives  to  spacecraft 

TYPICAL  RELIABILITY  CURVES 
FOR  (3)  SPACECRAFT 

One  of  the  first  and  most  important  aspects 
of  lifetime  incentives  is  that  perfection  la 
in  fact  possible/attainable,  so  that  for 
perfect  work  the  contractor  receives  high 
rewards.  But  probabilistically  speaking, 
each  spacecraft  is  imperfect  by  design 
and  contracts  don't  Impose  such  require¬ 
ments  of  assured  performance  perfection 
even  if  it  were  theoretically  posslble- 
which  I  believe  it  is  not  due  to  the 
nature  of  the  complex  system  being 
purchased . 

COMPUTER  RUN-SINGLE  PROGRAM 
SIMULATION  RESULT 

With  the  eliability  taken  as  a  constant 
assumption,  and  with  a  typical  3  space¬ 
craft  system  and  incentive  (orbital) 
structure  of  +  or  minus  8%  profit,  we  can 
simulate  the  overall  net  profit  which 
results  on  a  single  program  to  evaluate 
just  how  closely  profits  follow  quality. 

In  the  first  case,  the  profit  totals  +4% 
(for  the  simulated  program  and  incentive 
package,  including  a  launch  failure 
resulting  in  j  performance  incentives 
payoff  for  that  bird. 


However,  for  the  identical  contractor  per¬ 
formance  inputs,  the  incentive  results 
(outputs)  can  and  do  vary  widely  under 
the  simulation  of  actual  performance. 
Remember,  this  is  based  upon  a  constant 
theoretical  quality  of  product.  The 
point  is  that  the  inherent  uncertainties 
of  the  basic  product,  coupled  with  the 
typical  incentive  plan,  and  for  a  small 
quantity  sample  (3),  clearly  produces 
erratic  results  ranging  from  maximum 
profit  to  about  a  breakeven.  In  short, 
in  the  real  world,  the  theoretical 
advantages/trade-of f s  just  aren't  tliere. 

COMPUTER  RUN-TABLE,  NET  TIME  DISCOUNTED 

Now,  if  you  put  even  the  realized  prof  its 
in  their  lengthy  time  perspective,  those 
fees/profits  are  halved  or  more  in 
ef fectivenos'  “^rom  today's  standpoint. 
Moreover,  aer  deduction  must  be 

made  to  reo  these  paper  profits  to  net 
profits  for  taxes,  unallowables,  etc. 

Perhaps  it  is  a  good  thing  contractors 
don't  appear  to  pay  too  close  attention 
to  incentives  after  all ...  Perhaps  in¬ 
centives  are  like  tlie  mirage,  lovely  to 
look  at  from  afar  but  of  no  substance 
in  reality. 

ARMING  AMERICA  (RONALD  FOX) 

Another  fundamental  misconception  as  to 
profits  exists  in  the  present  day  dis¬ 
cussions  regarding  cost  saving  investment 
in  the  Defense  Industry  and  Profit  Policy 
to  encourage  same. 

COST  REDUGTl ON  INVESTMENT-EXAMPLE 

Ron  Fox  uses  an  example  of  how  cost-based 
profit  policy  effectively  precludes  con¬ 
tractors  from  making  cost  saving 
investments  because  to  do  so  reduces  their 
profits  which  are  based  on  costs.  Tliis 
misleading  example  supplied  by  the  industry 
in  1971,  is  unfortunately  either  incorrect 
or  oversimplified  in  that  the  added  coe l  : 
incurred  by  the  new  $8  Million  investment 
are  not  charged  against  the  analysis  (the 
$8  Million  investment  must  save  a  gross 
of  $18  Million  to  save  a  net  of  $10  Million) 


435 


1S)79  GAO  REPORT-ON  DEFENSE 
INDUSTRY  PRODUCTIVITY 

Obviously,  productivity  resulting  from 
efficient  new  investment  should  be  encou¬ 
raged,  and  GAO  recommends  new  profit 
structures  to  do  this,  but  at  the  same 
time  retaining  a  dumb  offset  principle  so 
that  no  more  profit  in  total  can  be 
obtained;  let's  see  by  example  what  would 
happen  and  what  then  should  be  done  to 
recognize  a  basic  dilemma... 

REAL  COST  REDUCTION  INVESTMENT 

If  we  build  on  Ron  Fox's  example,  and  use 
the  parameters  of  an  actual  case  from  the 
Investment  '76  Study  (part  of  Profit  '76), 
we  see  a  corrected  set  of  investment  and 
net  profit  results-again  without  compli¬ 
cations  of  overrun.  Using  the  profit  rate 
on  capital,  the  corrected  result  clearly 
penalizes  a  contractor  if  a  constant  10% 
profit  is  obtained  on  cost  after  Che  cost 
reduction.  Instead  of  14.3%,  the  return 
is  now  only  9.94%  on  capital,  a  dumb  move. 
In  column  2,  if  we  use  instead  the  "Profile 
Mill"  investment  of  $1.8M  and  attendant 
cost  savings  from  Investment  '76,  this 
smaller  investment  with  10%  cost-based 
profit  returns  a  reduced  capital  profit 
of  12,88%,  another  dumb  result.  In  column 
3  we  reverse  engineer  the  Column  2  profit 
to  Just  maintain  the  prior  14,3%  Return 
on  Capital  and  solve  for  contract  going-in 
amount  (and  rate)  needed  to  retain  capital 
profitability  parity.  The  rate,  in  this 
case  is  10.85%  without  benefit  of  time 
discount  correction  factoring  (which  would 
raise  it  about  double  the  increase). 
Carefully  note  that  the  price  to  the  U.S. 
Government  in  this  case  has  been  reduced 
from  110  to  108.42,  but  face  value  of 
profit  has  increased  8'2%. 

BEFORE/ AFTER-10%  COST  SAVINGS 

If  we  test  to  see  what  is  needed  to  bring 
about  a  larger  cost  savings  investment, 

10%  in  this  example,  a  constant  10%  Rate 
of  Profit  (on  cost)  drastically  slashes 
Return  on  Capital  to  8,71%,  whereas  a  43*2% 
Profit  Rate  increase  is  required  (before 
discounting)  to  just  retain  parity  on  a 
Rate  of  Return  on  assets  basis.  Note  that 
this  translates  into  something  less  than  a 
6%  net  lower  price  to  the  government. 


INCREMENTAL  CASH  FLOW  EXAMPLE 

The  point  is  that  GAO's  perception  of  con¬ 
stant  profit  percentages  on  cost  cannot 
begin  to  cope  with  the  economic/ financial 
realities  of  investment  motivation  in  a 
practical  way.  More  capital  investment 
requires  more  (net)  profitability  to  be 
a  real  motivator;  otherwise,  that  key 
Indicator  (ROAE)  of  management's  perfor¬ 
mance  will  be  degraded,  in  direct 
conflict  with  ever  increasing  goals  for 
improved  financial  results. 

BILL  PERRY'S  FY  '81  CONTRACT 
FINANCING  REMARKS 

At  the  policy  level,  the  criticality  of 
cash  flow  in  motivating  cost  effective 
investment  has  been  acknowledged,  and 
recently  HQ  AFSC  has  instigated  a  contract 
financing  survey  to  determine  improvements 
in  contract  financing  which,  if  imple¬ 
mented,  can  go  a  long  way  toward  solving 
some  difficult  problems  for  both 
government  and  industry. 

GRAPHICS  OF  TYPICAL  PERFORMANCE 
INCENTIVE  TIMING 

Performance  incentives  come  late  in  the 
game,  and  are  not  directly  connected  via 
the  present  day  incentive  or  product 
structure  to  deterministic  payoff  results. 

CASE  2-COMPUTER  RESULTS-10  CASES 

This  is  clearly  illustrated  by  the  random 
results  under  a  small  sample  program  of 
high  technical  uncertainty  coupled  with 
today's  incentive  structure,  as  noted  in 
the  various  studies  on  incentive  con¬ 
tracting,  there  is  far  more  theory  than 
results  in  this  field... 

ANALYSIS  METHODS-REVISITED 

The  government  typically  uses  a  nearly 
extinct  form  of  profit  and  incentive 
analysis-undiscounted  total  prof its-which 
is,  as  shown,  not  particularly  meaningful 
to  anyone  else.  The  industry,  on  the 
other  hand,  must  deal  with  a  far  lower  and 
more  realistic  aspect  of  profitability, 
including  the  time  discount  factor  and 
cash  flow  aspects,  net  of  taxes  and 
unallowable  costs. 


PROFITS  AND  PRICES 

If  costs  are  to  be  reduced  through  in¬ 
creased  productivity,  it  is  clear  that 
real,  not  just  face  amounts,  (of)  profit¬ 
ability,  must  rise  to  encourage  this 
ultimate  result.  Low  profits  are  not  in 
the  government's  best  interest,  nor  ours, 
and  much  needed  real  improvement  must  be 
made  if  the  desired  policy  objectives 
are  to  be  transformed  into  reality. 


DOD  POLICY  IN  CURRENT  ENVIRONMENT 


“THERE  IS  A  NEED  FOR  MAJOR  IMPROVEMENT  IN  THE 
RELATIONSHIP  BETWEEN  THE  DEPARTMENT  OF  DEFENSE 
AND  DEFENSE  CONTRACTORS.  " 

"WE  MUST  ATTRACT  THE  BEST  AND  MOST  EFFICIENT 
CONTRACTORS  TO  DO  BUSINESS  WITH  THE  DOD.  IN  ORDER 
TO  DO  SO  WE  MUST  .  .  . 

•  COMPENSATE  HIM  FOR  THE  USE  OF  HIS  RESOURCES 

•  REFLECT  THE  DEGREE  OF  RISK  INVOLVED 

•  PROVIDE  ADEQUATE  RETURN  TO  THE  SHAREHOLDERS 

•  ENCOURAGE  INVESTMENT  FOR  THE  MODERNIZATION 
AND  INCREASED  PRODUCTIVITY  OF  HIS  FACILITIES." 


EI^VIRONtilENTCeNeRALLY  GETTING 
•  MORE  SEVERE 

V  tNA^O^RIATE  TYPE  OF  CONTRACTS 
MNITIVE  INCENTIVES 
INCREASED  AUDIT  SURVEILLANCE 

General  accountinc  office  surveys 

PROFIT  LIMITING  LEGISLATION 
>>  MARKET  SIZE  uncertainty  V 

OVERHEAD  MANAGEMENT 
C/SCSC 

COST  ACCOUNTING  standards  \  \ 

CONGRESSIONAL  INVESTlldfTlO^ 


yrii 


kiness 


cc  PAirriv  ' 


ASPR  3  1300  CAS  414  CONTRACTOR  ASPR  3  808  W  G  L 


RETURN  ON  ASSETS  EMPLOYED 


HOAE  ISUNtITIVE  TOtOTM  RE  TURN  ON  SALES  (MAROINI  AS  WILL  AS  ASSETS  EMPLOYED 


PROEIT  AUETS 

SALES  AFTER  TAX  EMPLOYED  ROS  »  TURNOVER  -  ROAE 


RETURN  ON  SALES 
THE  CONTRACTORS'  VIEW  OF  PROFIT 


UNALLOWABLE  COSTS 


INTEREST 

X  INDEPENDENT  RESEARCH  AND  DEVELOPMENT 
^  BIDDING  AND  PROPOSAL  EXPENSE  CEILINGS 
'  ADVERTISING 
^  CONTRIBUTIONS 

COST  OF  ACQUISITIONS  AND  MERGERS 
LEASE  COSTS  WHICH  ARE  GREATER  THAN 
,  OWNERSHIP  COSTS 

CAS  STANDARD  CORALLARY  COSTS  ALSO 
'  UNALLOWABLE 

NEGOTIATED  DISALLOWANCES 


COST 

S100K 

FEE 

8 

SALES  VALUE 

108 

FEE.  AWARD  ON  COST 

1 

■  0%  1 

LESS; 

CONVERSION  TO  RETURN  ON  SALES 

•o.< 

7.4 

OVERRUN  OF  10  PERCENT  NO  FEE 

-0.6 

66 

UNALLOWABLE  COSTS 

1.0 

56 

NEGOTIATED  DISALLOWANCES 

09 

49 

FEDERAL  INCOME  TAX 

-2.4 

RETURN  ON  SALES 

1  «  J 

NEGOTIATED  PROFIT  RATES 
FIXED  PRICE  INCENTIVE  CONTRACTS 


1976 

1977* 

CHANGE 

ARMY 

12.1% 

12.5% 

+  .4% 

NAVY 

12.5% 

11.3% 

•1.2% 

AIR  FORCE 

11.2% 

12.0% 

+  .8% 

DOD  TOTAL 

11.6% 

11.8% 

+  .2% 

•  INCLUDES  CAS  414  0 

•  INCLUDES  CAS  414  COST  OF  MONEY 


OVERRUNS 

NOT  EXCLUSIVE  TO  DEFENSE  CONTRACTORS 

GAO  REPORT:  59  NON  DoD  PROJECTS  $780M  OVERRUN 

OTHER  OVERRUNS: 

•  ALASKAN  PIPELINE 

•  RAYB-'JKN  OFFICE  BUILDING 

•  BART 

•  NEW  ORLEANS  SUPERDOME 

•  METRO 

•  QUEEN  MARY,  LONG  BEACH 

•  CANADIAN  OLYMPICS 
P  YANKEE  STADIUM 


AWARD  FEE 

AF  PROGRAM  MANAGER'S  VIEWPOINT 
Col.  Donald  W.  Henderson 
Deputy,  Navigation  Systems 
Space  Division 

AWARD  FE^ 

FROM  A  PROGRAM 
MANAGER’S  NARROW, 

BIASED  VIEWPOINT 

V 

BUT  SAGACIOUS 


WHATISINCENTIVIZED? 

•  COST/SCHEOULE/PERFORMANCE 

•  RESPONSIVENESS 

•  SUBCONTRACTOR  MGT 

•  TEST  PROGRAM  MANAGEMENT 

•  C/SCSC  MANAGEMENT 

•  ANYTHING  THE  PGM  MGR  WANTS 


AWARD  FEE  —  A  PERCEPTION 
DICHOTOMY 


PROFIT 


CHALLENGE 


TO  MAKE  AWARD  FEE  A  MOTIVATOR  AT  ALL  LEVELS 


EXAMPLE:  CPAF  CONTRACT 

•  3%  BASE  FEE 

•  12%  AWARD  FEE 


WHY  A  PROGRAM  MANAGER  LIKES  AWARD 
FEE 


•  CONTROL 

•  GETS  ATTENTION 

•  CAN  CHANGE  YOUR  MIND 

•  JUST  A  LITTLE  MONEY  GOES  A  LONG  WAY 


8—12%  ■  EMPLOYEES  SHARE 
12—15%  •  EMPLOYEES  GET  ALL 


GPS  AWARD  FEE  EXAMPLES 


•  THE  ONLY  INCENTIVE  THAT  WORKS  THE  WAY  IT  SHOULD 

NAVSTAR  GPS  AWARD  FEE 
EXPERIENCE 

•  IT  WORKS 

•  I  LIKE  IT 


CONTRACT 

BASE  FEE 

OTHER 

FPIFiAF 

6.4%  TOT 

2% 

+ 10% 

$100  MIL  TOT 

-  6% 

CPAF 

3% 

12% 

$4  MIL  TOT 

•  BENEFITS  JUSTIFY  EFFORT 


CPAF 

$95  MIL  TOT 


3% 


9 


NAVSTAR  GPS  AWARD  FEE 


PROBLEMS  WITH  AWARD  FEE 


•  ADMINISTRATIVE  WORK  LOAD 

8  FPIF/AF 
2CPAF 

•  CONTRACTOR  PERCEPTION 
•  MOTIVATOR/DE-MOTIVATOR 

NAVSTAR  GPS  AWARD  FEE 

20  TOTAL  CONTRACTS 

GPS  AWARD  FEE  RANGE 


•  91%  AWARD  FEE  ($91,700) 

•  10%  A.F.  +  3%  BASE  =  13%  PROFIT 


•  0%  AWARD  FEE 


TOTAL  CONTRACT  $  =  $722  MILLION 


•  0%  A.F.  +  4%  BASE  =  4%  PROFIT 


AWARD  FEE  DETERMINATION 


AWARD  FEE  FROM  A  PROGRAM 
OFFICE  VIEWPOINT 


© 
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Richard  M.  Randall 

Director,  Business  Management 

McDonnell  Douglas  Astronautics  Company  -  West 


There  are  some  areas  of  relevant  back¬ 
ground  material  that  ouaht  to  be  recalled 
briefly,  to  set  the  stage  for  presentino 
an  industry  viewpoint  on  Award  Fee 
Incentives. 

First,  let's  glance  at  the  DAR  provision 
setting  forth  the  criteria  for  use  of  the 
Cost  Plus  Award  Fee  type  of  contract.  The 
main  features  of  DAR  3-405.5  are; 

°  Contract  not  susceptible  to  finite 
measurements  of  performance  for  struc¬ 
turing  incentive  contracts. 

°  Fixed  fee  (not  to  exceed  3%)  plus  award 
fee  (total  not  to  exceed  maximum  fixed 
plus  award  of  ]0%,  except  that  15X  may 
be  authorized  in  contracts  for  experi¬ 
mental,  developmental,  or  research  work) . 

°  Motivate  excellence  in  such  areas  of 
performance  as  quality,  timeliness, 
ingenuity,  and  cost  effectiveness. 

°  Subjective  evaluation  by  Government 
based  on  performance  judged  in  theliaht 
of  criteria  set  forth  in  contract. 

°  Although  contract  should  provide  con¬ 
tractor  an  "opportunity  to  comment  on 
evaluation,"  the  decision  is  unilateral 
and  non-disputable. 

°  Applicable  to  level  of  effort  contracts 
for  performance  of  services  and  to 
contracts  which  would  have  been  of 
another  type  if  performance  objectives 
could  be  expressed  in  advance  by  defin¬ 
itive  milestones,  targets  or  costs 
susceptible  of  measuring  actual  perfor¬ 
mance. 

Award  Fee  Incentives  were  discussed  during 
last  year's  Mission  Assurance  Workshop 
and  observations  recorded  then  are  proba¬ 
bly  germane  today.  Thus,  let's  consider 
several  of  those  observations; 

°  USAF  acquisition  study  reveals  that 
industry  unanimously  favors  award  fees 
because; 

They  like  a  "report  card"  and  it  is 
extremely  useful  to  program  managers 
and  award  fees  force  stronn  communica¬ 
tion  links  between  the  contractors  and 
the  Government. 


°  In  R&D  contracts,  incentivizino  admin¬ 
istrative  features  may  cause  the  con¬ 
tractor  to  emphasize  these  areas  at 
the  expense  of  performance. 

°  The  potential  motivational  value  of 
award  fees  should  be  balanced  against 
the  resulting  administrative  cost  and 
effort. 

Project  Vanguard  has  considered,  and  made 
impressions  on,  contract  methods  and 
practices.  Award  fees  got  passing  atten¬ 
tion  in  General  Slay's  April  1979  message 
to  industry  which  referred  to  the  follow¬ 
ing  areas  of  concern  in  contractinn; 

°  Move  towards  commercial  contract 
methods  and  practices ; 

°  FFP  for  production. 

In  R&D  selectively  use  fixed  cost, 
best  efforts  contracts  with  award 
fee  provisions  (competitive  proto¬ 
typing). 

^  Use  warranties  covering  performance, 
reliability  and  maintainability, 
where  requirements  and  specs  are 
well  defined. 

°  Greater  use  of  past  performance  for 
selection  criterion. 

°  More  competition,  less  sole  source. 

However,  award  fees  did  not  receive 
specific  mention  in  the  General's 
March  4,  1980,  updating  message  to 
industry.  For  the  record,  he  did  single 
out  for  reference,  the  following  contrac- 
tinq  initiatives; 

°  More  competition. 

”  Increased  use  of  fixed-price  contracts. 

"  Tighte’" incentive  contracts. 

"  Greater  use  of  warranties. 

"  Heavy  weighting  of  past  performance  in 
source  selection. 

°  Use  of  draft  RFP's. 

°  Multi-year  contract! no  and  "a  few  others. 


The  topics  I  chose  to  discuss  during  my 
remarks  include: 

°  Use  of  award  provisions  to  add  to  objec¬ 
tive  incentives  to  enhance  mission 
assurance. 

°  Use  of  "over-and-above"  award  provisions 
with  CPIF,  FPI  or  FFP  contracts  (DAR 
3-405. 5(h)). 

°  SGS-II  as  an  example  of  combined,  moti¬ 
vational  incentive  features. 

DAR  3-405. 5(h),  "Other  Application  of 
Award  Fee  Provisions"  provides  the  follow¬ 
ing  guidance  and  admonition; 

”  DAR  3-405. 5(h),  "Other  Application"  of 
Award  Fee  Provisions. 

°  Use  to  motivate  and  reward  for  manage¬ 
ment  performance  "over  and  above"  that 
which  can  be  objectively  measured  and 
incentivized.  Areas  for  application 
include: 

Logistics  support 

Duality 

Timel iness 

Cooperation 

Ingenuity 

Cost  effectiveness 

”  Don't  use  award  fee  provisions  in  con¬ 
junction  with  "other  types"  of  contracts 
when  the  administrative  effort  or  costs 
for  evaluation  exceed  the  benefits  to 
be  derived. 

In  my  opinion,  this  utilization  of  the 
award  fee  incentive  offers  more  potential 
for  meaningful  motivation  than  the  DAR 
provisions  dealing  with  the  use  of  CPAF 
contracts  generally. 

The  SGS-II  contract  with  Systems  Division 
contains  what  I  consider  to  be  a  simple, 
effective  combination  of  commercial 
features,  fixed-price  format,  no-nonsense 
schedule  and  performance  incentives  and 
"over-and-above"  award  fee  that  can  be 
earned  by  superior  program  management. 

Here  is  a  digest  of  the  contracts  incen¬ 
tive  and  award  features: 

°  Firm  Fixed  Price  contracts. 


°  Incentives  -  negative  only  -  established 
on  schedule  and  mission  performance. 

°  Schedule: 

Delivery  of  hardware  to  VAFB  and  readi¬ 
ness  for  launch  (graduated  depending  on 
days  late  to  schedule)  (2%). 

°  Mission  performance: 

All  or  nothing  penalty  based  upon  per¬ 
formance  against  target  performance 
parameters  (10%). 

°  Awards  established  on  management  in 
areas  of: 

A)  Program  management 

B)  Program  integration 

C)  Test  and  evaluation 

0)  Launch  support  services 

E)  Systems  effectiveness 

°  Award  fee  assigned  up  to  6  milestones 
(depending  on  option  exercise),  each 
about  9  months  apart  with  allocations 
of  from  10%  to  25%  of  total  pool  (3%). 

Finally,  I  must  take  advantage  of  the 
opportunity  to  make  some  personal 
comments  based  upon  my  experiences  with 
award  fee  contracts  with  a  number  of 
Government  agencies: 

°  They  ^  get  management's  attention  and 
may  motivate  in  wrong  directions. 

°  Evaluators  have  downgraded  awards  for 
reasons  outside  the  purview  of  stated 
eval uation  criteria. 

°  "Report  cards"  may^  cause  confusion  and 
bitterness . 

°  "Cooperation"  is  important,  but  too 
much  can  be  expensive. 

"Working  for  tips"  can  place  form  over 
substance.  (Good  service  does  not 
compensate  for  a  bad  meal.) 

°  Define  what's  really  important  and  put 
the  bucks  there. 


VALUE  ENGINEERING 
Mr.  O.J.  Vogl 

Manager,  Value  Engineering 
Hughes  Aircraft  Company 


VALUE  ENGINEERING  AND  DESIGN-TO-COST 
INCENTIVES  > 

Because  of  the  successful  application  of 
Value  Analysis  in  shipbuilding,  the 
government  introduced  in  the  Armed  Ser¬ 
vices  Procurement  Regulation  (ASPR) 
clauses  to  encourage  contractors  to  sub¬ 
mit  contractual  changes  that  would  re¬ 
duce  government  costs.  The  clauses 
offered  contractors  a  share  of  the 
savings  effected  by  an  accepted  pro¬ 
posal.  The  development  and  implementa¬ 
tion  costs  of  the  changes  are  deducted 
from  the  gross  savings  to  determine  the 
net  savings  to  be  shared. 

The  ASPR  -  now  changed  to  Defense  Acqui¬ 
sition  Regulation,  DAR,  defines  the 
Value  Engineering  Change  Proposal  (VECP) 
as:  A  change  to  the  contract  that  re¬ 
sults  in  reducing  government  costs.  A 
contractor  participates  in  the  sharing 
of  accepted  VECPs  in  accordance  with 
the  percentage  associated  in  the  DAR 
with  the  type  of  clause  and  the  type  of 
contract . 

Two  types  of  Value  Engineering  Clauses 
are  specified: 

1.  A  Value  Ehglneerlng  Program  Require¬ 
ment  Clause. 

The  contractor  is  paid,  by  line  item 
in  the  contract,  to  perform  Value 
Engineering.  A  formal  organization 
and  program  are  required;  ther^  is 
a  Military  Specification  to  be  met. 


Design-to-Cost  Interface  with  Value 
Engineering 

The  DAR  Value  Engineering  clauses  do  not 
enccmpass  the  same  scope  as  the  Design- 
to-Cost  clauses.  D-T-C  is  applied  to 
the  contract  elements  -  work  break-down 
structure  -  to  have  the  contractor  meet 
a  specified  cost  for  the  manufactured 
system.  The  changes  made  by  the  con¬ 
tractor  are  within  the  limits  of  the 
contract.  VECPs  are  all  out  of  scope 
changes  -  changes  to  the  contract  to 
reduce  costs.  Contractors  employ  the 
Value  Engineering  methodology  to  meet 
their  D-T-C  goal,  but  such  changes  do 
not  require  contractual  changes. 

To  avoid  duplication  of  awards  to  con¬ 
tractors  one  of  two  methods  may  be 
used: 

1.  Each  accepted  VECP  reduces  the  D-T-C 
goal  by  the  savings  proposed  in  the 
VECP. 

or 

2.  No  VE  award  is  given  the  contractor 
until  he  has  met  his  D-T-C  goal. 

After  the  D-T-C  goal  has  been 
achieved,  the  contractor  shares  in 
subsequent  savings. 

GAO  Report 

In  1979,  the  Government  Accounting  Office 
issued  a  report  showing  the  short ccmlngs 
of  the  services  savings  under  the  DAR  VE 
clauses.  In  response  the  Secretary  of 
Defense  issued  a  memorandum  establishing 
an  annual  goal  for  VECP  savings  in  each 
service.  This  goal  is  7/10%  of  each 
service's  procuranent  dollars. 


2.  A  Value  Engineering  Incentive  Clause: 
The  contractor  is  not  required  to 
perform  Value  Engineering  studies 
but  is  encouraged  to  do  so.  He  is 
not  paid  to  do  it,  but  does  receive 
a  larger  share  in  the  savings  of 
accepted  Value  Engineering  Change 
Proposals  (VECP). 
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VALUE  ENGINEERING  CHANGE  PROPOSAL 
(VECP) 


SHARING  OF  V  E  SAVINGS  ACCORDING 
TO  TYPE  OF  CONTRACT 


VECP 

•  A  VECP  IS  A  COST  REDUCTION  PROPOSAL 
SUBMITTED  UNDER  A  VE  CONTRACT  CLAUSE 

THAT  IF  ACCEPTED  WILL: 

•  REQUIRE  A  CHANGE  TO  THE  CONTRACT 

•  REDUCE  THE  OVERALL  COST  TO 
THE  CUSTOMER 

A  VECP  MUST  INVOLVE  SOME  CHANGE  IN 
THE  CONTRACT: 

•  SPECIFICATIONS, 

•  PURCHASE  DESCRIPTION 


T  'PEOf  CONTRACT 

•INCENTIVE 

CLAUSE 

GOVT/CO.'JTR 

•PROGRAM 

CLAUSE 

GOV  T/CONTR 

FlX30FF«:C£  (OTHER  THAN  INCENTIVE) 

so/so 

72/26 

Fixed FF.CE  -tive  (FPiior 

COST  PLUS  INCfcHTJVL  FEE  (CPIF) 

65/35 

80/20 

COST  PLUS  AWARJ  FEE  ICPAF) 

76/25 

65/15 

COST  R£l..1QURSEr.iEi.T  (OTHER 
tha:i  criF  andcpafj 

76/25 

85/15 

COfjT, FACTOR'S  SHARE  OF  COLLATERAL  SAVINGS 
IS  20%  FOR  AVERAGE  OR  TYPICAL  YEAR  OF  USE. 

•£h/  filWG  IS  SAME  FOR  INSTANT  AND  FUTURE  ACQ 


VECP  SAVINGS 


INSTANT  CONTRACT  *  cost  peduction  on  contract  under  which  VECP  is  submitted. 


CONCURRENT  COtiTRACT  -  cost  reduction  on  other  existing  contracts  affected  bv  the  VECP 


FUTURE  CONTRACTS  *  cost  reductions  on  all  quantities  not  under  contract  on  date  of 
VECP  ACCEPTANCE. 


COLLATERAL  SAVINGS  -  Government  savings  in  maintenance,  operation,  logistic  suppwkt.  '.r 
Government  furnished  propertt. 


•  STATEMENT  OF  WORK 

•  GENERAL  AND  SPECIAL  PROVISIONS 

•  OTHER  I TEMS 

A  VECP  MUST  INVOLVE  SOME  CHANGE  IN 
THE  CONTRACT: 

•  SPECIFICATIONS, 


GOVERNMENT/CONTRACTOR 

BENEFITS 


CONTRACTOR 

INCREASE  EARNINGS  ON 
CURRENT  AND  FOLLOW  ON 
CONTRACTS 
INCORPORATE  NEW 
TECHNOLOGY 
ENHANCE  COMPETITIVE 
POSITION 

DELETE  UNNECESSARY/ 
COSTLY  REQUIREMENTS 
EXPAND  MARKET  POTENTIAL 
REDUCE  OVERRUN 
IMPROVE  CASH  FLOW 


GOVERNMENT 

REDUCE  CONTRACT  COST 
COMMITMENTS 

PROVIDE  FUNDS  FOR  NEEDED 
CHANGE 

PERMIT  TECHNOLOGY 
UPGRADE 

PROVIDE  INCREASE  IN 
PRODUCT  CAPABILITY 
REDUCE  COST  OF  OPERATION 
AND  SUPPORT 

REDUCE  OR  ELIMINATE  NON 
COST  EFFECTIVE  REQUIRE 
MENTS 

KEEP  PROGRAM  SOLD 


•  PURCHASE  DESCRl PTI ON 

•  STATEMENT  OF  WORK 


GAO  REPORT 


POTENTIAL  FOR  VE  SAVINGS  IN  DoD;  $250  MILl.lON/VEAP 


ACTUAL  -  IRS!  -  « 


»700  MILLION 


S  PROGRAMS;  S  21  MILLION  IN  IR7& 

I  PROGRAM  ;  t  SO  MILLION  IN  LAST  6  YEARS 


•  GENERAL  AND  SPECIAL  PROVISIONS 


MOST  MAJOR  D9D  PROGRAMS  HAD  NO  VE  SAVINGS. 

IN  1R7S  ONLY  24  (OUT  OF  881  MAJOR  DoD  PROGRAMS  RtPOPl  ED  VE  SAVINGS 


•  OTHER  I TEMS 


ARMY  10  PROGRAMS 

AIR  FORCE  10  PROGRAMS 

NAVY  4  PROGRAMS 


DoD  SOFTWARE  BUDGET 


DESIGN  TO  COST  FOR  EMBEDDED 
COMPUTER  SYSTEMS 

Mr.  John  Munson 

V.P.  Corporate  Software  Engineering 
Systems  Development  Corporation 


APPLICATION  OF  DESIGN-TO-COST  CONCEPT 


•  OPERATIONAL  CAPABILITY  DETERMINEO  BY  AVAILABLE  DOLLARS 


•  COST  AS  A  FACTOR  EQUIVALENT  TO  TECHNICAL  CAPABILITIES 


•  ESTABLISHMENT  OF  OESIGN-TO-UNIT-PRODUCTION-COST  GOALS 


•  TOTAL  LIFE-CYCLE  COST  AS  A  SELECTION  CRITERIA 


APPLICATION  OF  OESIGN-TO-COST  CONCEPT 


DoD  SOFTWARE 
EXPENDITURE 


•  OPERATIONAL  CAPABILITY  DETERMINEO  BY  AVAILABLE  DOLLARS 


•  COST  AS  A  FACTOR  EQUIVALENT  TO  TECHNICAL  CAPABILITIES 


40% 

ACQUISITION 

$1.2B 


•  ESTABLISHMENT  OF  DESIGN-TO-UNIT-PROOUCTION-COST  GOALS 


•  TOTAL  LIFE  CYCLE  COST  AS  A  SELECTION  CRITERIA 


SOFTWARE  FOR 

EMBEDDED  COMPUTER  SYSTEMS 


60% 

MAINTENANCE 

$1.8B 


$3  BILLION 


•  SOFTWARE  PROVIDES  MAJOR  SYSTEM  FUNCTIONALITY 

•  GENERALLY  "ONE-OF-A-KIND" 

•  APPLICATIONS  SOFTWARE  NORMALLY  UNIQUE 

•  NO  PRODUCTION  CYCLE  lAT  LEAST  FOR  SOFTWARE) 

•  SOFTWARE  ALWAYS  ON  CRITICAL  PATH  AND  IS 
HIGHEST  RISK  ELEMENT 


CONCLUSION  RE; 

DTC  IN  EMBEDDED  COMPUTER  SYSTEMS 


THE  LIFE-CYCLE  COST  CRITERIA  IS 
CLEARLY  THE  MOST  REALISTIC  SELECTION 
ALTERNATIVE  FOR  ACQUIRING  EMBEDDED 
COMPUTER  SYSTEMS 


•  LIFE-CYCLE  COSTS  A  MAJOR  SOFTWARE  ISSUE 


.  .  .  BUT  IS  IT  POSSIBLE? 


NOT  IN  ANY  PRACTICAL  MEASURE  TODAY 


WE  CAN  ONLY  "BUY"  PERFORMANCE! 

OUR  CONTRACTUAL  AND  TECHNOLOGICAL  TOOLS 
ARE  INADEQUATE  TO  MEANINGFULLY  PROCURE 
LIFE-CYCLE  COST  ATTRIBUTES 


CHARACTERISTICS  OF  SOFTWARE 


•  SOFTWARE  IS  INTANGIBLE— NEVER  EXISTS 
PHYSICALLY 

•  SOFTWARE  RELIABILITY  IMPROVES  WITH  USE 

•  CORRECT  S/W  NEVER  FAILS 

•  REPAIR  MEANS  CHANCE  BASELINE 

•  EASY  TO  CHANGE— HARD  TO  CHANGE— RIGHT 

•  SOFTWARE  FAILURE  IS  UNPREDICTABLE 


FOR  SOFTWARE.  LIFE-CYCLE  COST 
CONSIDERATION  MEANS  CONCERN 
FOR  THE  MODIFIABILITY  ATTRIBUTES 
OF  THE  DELIVERED  SYSTEM 


MODIFIABILITY  ISSUES 


1.  CONTRACTING  FOR  MODIFIABILITY 

2.  TECHNIQUES  FOR  BUILDING-IN 
MODIFIABILITY 

3.  POST-DELIVERY  TEST  FACILITIES  & 
SUPPORT  TOOLS 

4.  DOCUMENTING  FOR  EASE  OF 
MODIFIABILITY 

5.  QUALITY  ASSURANCE  OF  MODIFIABILITY 


MODIFIABILITY  STARTS 
WITH  THE  CONTRACT 


•  SET  MODIFIABILITY  AS  CONTRACT  REQUIREMENT  & 
INCENTIVIZE  WITH  AWARD  FEE 

•  REQUIRE  S/W  MODIFIABILITY  DEVELOPMENT  PLAN 

•  DEFINE  DESIGN  TECHNIQUES 

•  SET  PROGRAMMING  STANDARDS 

•  SPECIFY  MANAGEMENT  CONTROLS 

•  REQUIRE  S/W  MODIFICATION  SUPPORT  PLAN 

•  TEST  FACILITIES 

•  SUPPORT  &  TEST  TOOLS 

•  REQUIRE  DELIVERY  OF  ALL  DEVELOPMENTAL  TOOLS, 
TEST  MATERIALS  &  DOCUMENTATION 


BUILDING  MODIFIABLE  SOFTWARE 


CLARITY  &  SIMPLICITY 


447 


TECHNIQUE 


REVIEWING  FOR  MODIFIABILITY 


•  EMPLOY  HIERACHICAL  CONTROL 
STRUCTURE 

•  ENFORCE  MODULAR  DESIGN 

•  UTILIZE  SINGULARITY  OF  FUNCTION 

•  ISOLATE  EXTERNAL  ENVIRONMENT 

•  LIMIT  DATA  ACCESS 

•  USE  CENTRALIZED  DATA  STRUCTURES 

SOFTWARE  "COST  DRIVERS" 

•  USER  INTERFACE 

—  DISPLAYS 
—  REPORTS 
—  INPUTS 

•  EQUIPMENT  CONFIGURATION 

•  LIMITS  &  CAPACITIES 

•  EXTERNAL  COMMUNICATION  FORMATS 

•  LACK  OF  PROBLEM  DIAGNOSTICS 

O  &  M  SUPPORT  CAPABILITIES 


•  PLANS  &  STANDARDS 

•  SYSTEM-LEVEL  DESIGN  REVIEWS 

•  DETAILED  [COMPUTER  PROGRAM] 
DESIGN  REVIEWS 

•  AN  ADEQUATE  TEST  PROGRAM 

•  SYSTEM-LEVEL  TESTING 

•  ACCEPTANCE  &  DELIVERY  INSPECTION 

SUMMARY 

•  THE  PRINCIPAL  DTC  ISSUE  IN  SOFTWARE  IS  MODIFIABILITY 

•  MODIFIABILITY  CANNOT  BE  OBJECTIVELY  SPECIFIED 

•  THE  MODIFIABILITY  CHARACTERISTICS  ARE  BUILT  IN 
DURING  DEVELOPMENT 

•  IF  YOU  DONT  SPECIFY  IT  IN  THE  CONTRACT.  YOU  WONT 
GET  IT 


•  ADEQUATE  EQUIPMENT 
RESOURCES  FOR  TEST 

•  DATA  SIMULATION  TOOLS 

•  ON-LINE  DIAGNOSTICS 


•  DATA  RECORDING 


AFSC  INCENTIVE  POLICY  AND  SPACE 
DIVISIONS  INCENTIVES 

Mr.  Harold  E.  Sharp 
Director  of  Pricing 
Space  Division 


RECENT  SPACE  DIVISION  CONTRACTS  WITH  PERFORMANCE  INCENTIVES  ($  IN  THOUSANDS) 


PROGRAH/CONTRACT 

DFSCRIPTION 

TYPE 

SHARE/ 

PERCENT 

%  PERFORMANCE 

INCENTIVES _ 

1.  DSP/ 

FPIF 

70/30 

+  4.7 

ONE  SENSOR 

T.P.  m 

C.P.  120% 

-12.0 

MONTHLY 

INCENTIVES 

YEARS 

TIME  OF 

PER  SPACECRAFT 

PAR  TOTAl 

PAYMENT 

NEGATIVE 

POSITIVE 

AMI  m 

AUI  m 

$55  36 

$114  36 

1/4  3 

FINAL 

ACCEPTANCE 

2.  DHSP 

FPIF  65/35  +21.2 

68 

48 

61  36 

3  4 

AT  END  OF  3 

3  SATELLITES 

T.P.  10.38%  -14.9 

YEARS  QUART¬ 

C.P.  120% 

ERLY  FOR  4TH 

YEAR 

3.  GPS/ 

FPIF/AF  65/35  +13.0 

32 

78 

40*54 

1/4  6^5 

QUARTERLY 

4  SATELLITES 

T.P.  11%  -15,0 

C.P,  119%  +  .AWARD.  FEE 

AWARD  FEE  45 

OF  +6.0 

DAYS  AFTER 

NOTIF, 

4.  DSP/ 

FPIF  70/30  +11.8 

19 

120 

87  36 

3  10 

FINAL 

RETROFIT  4 

T.P.  12%  -  16.2 

ACCEPTANCE 

SATELLITES 

C.P.  120% 

*  IN  ADDITION,  ONE  TIME  NEGATIVE  IfifEi!TIVE  OF  $776  TIITrL  Of’ER''TIONAL  CAPABILITY  IS  ASSESSED. 


CONTRACT  INCENTIVES 


Recommendations 


SUMMARY  OF 

ISSUES  AND  RECOMMENDATIONS 


1.  Contract  Incentives  -  High  Cost 
of  Money 

Issue 

The  time  value  of  performance 
incentives  is  diminished  by  the 
high  cost  of  money. 

Recommendations 

o  Continue  to  make  advance  pay¬ 
ments  of  incentives  with  payback 
provisions. 

2.  Value  Engineering 
Issue 

Customers  are  reluctant  to  fund 
value  engineering  programs. 

Recommendations 

o  Investigate  increasing  the  use  of 
V.  E.  programs  and  incentive. 

3.  Imbedded  Software 
Issue 

Existing  acquisition  regulations, 
procedures  are  inadequate  for  the 
acquisition  of  imbedded  computer 
software. 

Recommendations 

o  Investigate  ways  to  properly  in- 
centivize  software  contractors  on 
unique  aspects  of  software  per¬ 
formance. 

4.  Contract  Incentives  Guide 
Issue 

There  is  no  current  standard  guide 
for  incentive/award  fee  contract¬ 
ing. 


o  Form  a  joint  Ad-Hoc  committee 
(DOD,  NASA,  Industry'  to  write 
a  current  guide. 

5.  Contract  Incentives,  Award  Fee 
Criteria 

Issue 

Award  fee  criteria  are  not  always 
clearly  understood  and  agreed  to 
with  priorities  established.  Scor¬ 
ing  sometimes  considers  other 
criteria  than  those  agreed  to. 

Recommendations 

oKeep  award  fee  periods  short 
when  possible.  For  long  periods, 
hold  interim  status  meetings. 
Follow  the  award  fee  plan. 

o.  Award  Fee  Pool 
Issue 

AFSC  restricts  the  carry-forward 
of  unawarded  award  fee  pool  for 
later  consideration. 

Recommendations 

o  Pursue  modification  of  restrict¬ 
ive  regulations  to  allow  for  post¬ 
program  re-evaluation  to  award 
the  unawarded  fee. 


VALUE  ANALYSIS  OF  QUALITY 


A  BETTER  PRODUCT  AT  LOWER  COST 
H.  Dean  Voegtlen 
HUGHES  AIRCRAFT  CO. 


ABSTRACT 

The  nature  and  objectives  of  the  quality 
control  and  value  engineering  programs 
are  examined.  Value  engineering  is 
shown  to  be  a  powerful  tool  to  achieve 
overall  quality  objectives.  Some  results 
of  VE  studies  at  Hughes  Aircraft  Company 
and  in  other  industries  are  reviewed.  It 
is  shown  that  these  results  have  pro¬ 
duced  substantial  cost  reduction  as  well 
as  other  Important  quality  benefits. 

PROPERLY  APPLIED,  THE  SCIENTIFIC  METHODS 
used  in  value  engineering/analysis  will 
enhance  the  quality  of  products  or  ser¬ 
vices  supplied  in  today's  market  place. 
This  thesis  is  examined  through  a  com¬ 
parison  of  the  objectives  of  both  the 
Quality  and  Value  Engineering  programs 
in  general.  Specific  findings  that  sup¬ 
port  the  thesis  are  illustrated  through 
the  author's  experience  at  Hughes  Air¬ 
craft  Company. 

QUALITY  PROGRAM  OBJECTIVES 

A  good  definition  of  the  quality  func¬ 
tion  in  an  organization  was  stated  by 
A.  V.  Felgenbaum,  a  well-known  author, 
teacher,  and  consultant  in  the  quality 
field:  "Quality  Control  is  a  system  for 
administration  and  coordination  of  qual¬ 
ity  maintenance  and  quality  improvement 
efforts  within  the  organization  to  en¬ 
able  the  manufacture  of  a  product  at  the 
most  economical  level  that  will  allow 
full  customer  satisfaction". 

Some  of  the  key  words  are  underlined. 
Quality  maintenance  includes  settine 
standards  or  acceptable  quality  levels, 
measuring  conformance,  and  obtaining 
corrective  action.  Quality  improvement 
involves  the  setting  of  goals,  develop¬ 
ment  of  new  quality  methods,  instilling 
quality  mindedness,  and  those  related 
tasks  needed  to  raise  the  level  of  qual¬ 
ity  of  the  delivered  product.  The  final 


phrase,  "at  the  most  economical  level", 
is  the  real  challenge.  The  value  or 
worth  of  quality  must  equal  the  cost  of 
quality  or  there  will  be  unhappy  custom¬ 
ers.  Clearly,  the  quality  function's 
charter  must  include  a  strong  economic 
ingredient.  Too  much  quality  will  bear 
an  unacceptable  price;  too  little  quality 
will  result  in  failure  or  unacceptable 
performance  in  the  end-use  environment. 
How  can  a  delicate  balance  be  achieved? 

THE  VALUE  ENGINEERING  FUNCTION 

A  widely  used  definition  of  value  engi- 
eering  reads  as  follows: 

A  systematic  effort  directed  at  anal¬ 
ysis  of  the  functional  requirements 
of  systems,  equipment,  facilities, 
procedures,  and  supplies  for  the  pur¬ 
pose  of  achieving  the  essential  func¬ 
tions  at  the  lowest  total  cost  con¬ 
sistent  with  needed  performance,  re¬ 
liability,  maintainability,  etc. 

The  key  concepts  stated  or  implied  in 
this  definition  are  function,  worth, 
cost,  and  value.  Function  is  defined 
as  the  specific  purpose  or  use  intended 
for  a  product  or  portion  thereof.  In 
VE  studies  the  description  of  function 
is  reduced  to  the  simplest  accurate  ex¬ 
pression  employing  only  two  words:  a 
verb  and  a  noun.  For  example,  "support 
weight",  prevent  corrosion",  and  "con¬ 
duct  current"  are  typical  expressions  of 
function. 

Worth  is  the  least  expenditure  required 
to  provide  functions  needed  by  the  user. 
Worth  is  established  by  comparison.  One 
method  of  approximating  worth  is  by  de¬ 
termining  the  cost  of  a  functional  equiv¬ 
alent.  Worth  is  not  affected  by  the  con¬ 
sequence  of  failure.  For  example,  if  a 
bolt  supporting  the  wing  of  an  aircraft 
fails,  the  plane  may  crash.  Nevertheless, 
the  worth  of  the  bolt  is  the  lowest  cost 
necessary  to  provide  a  reliable  fasteninr. 

Cost  is  the  total  funds  needed  to  acquire 
and  use  the  specified  functions.  For  the 
seller  this  is  the  total  of  his  expendi¬ 
tures  in  connection  with  the  product.  The 
Department  of  Defense  tends  to  define 
this  as  total  cost  of  ownership  or  life- 
cycle  cost.  It  includes  acquiring  the 
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product,  introducing  it  into  the  inven¬ 
tory,  operating  it,  and  supporting  it 
through  its  usable  life. 

Value  is  the  relationship  of  worth  to 
cost  as  seen  by  the  user.  This  ratio  is 
the  principal  measure  of  value: 

,,  ,  ,  Worth  Utility  ,,, 

Value  Index  =  — — -  =  ^  (1) 

Cost  Cost 

Value  may  be  increased  by  (l)  improving 
utility  with  no  change  in  cost,  (2)  re¬ 
taining  the  same  utility  for  less  cost, 
or  (3)  combining  improved  utility  with 
lowered  cost.  Optimum  value  is  achieved 
when  all  utility  criteria  are  met  at  the 
lowest  overall  cost. 

The  similarity  of  concept  between  quality 
control  and  value  engineering  is  obvious. 
Both  programs  are  aimed  at  meeting  custo¬ 
mer  requirements  at  the  lowest  possible 
cost.  While  the  quality  program  is  com¬ 
mitted  to  preservation  of  acceptable 
standards, the  VE  program  is  continually 
probing  to  find  out  if  these  standards 
can  be  met  at  a  lower  cost.  While  qual¬ 
ity  activities  differ  from  those  of  value 
engineering, it  is  safe  to  say  that  a 
product  of  acceptable  quality  as  defined 
above  will  also  have  good  value.  Value 
engineering  can  be  thought  of  as  a  tool 
to  help  meet  overall  quality  objectives. 

VALUE  ENGINEERING  AT  HUGHES  AIRCRAFT 
COMPANY 

For  over  20  years  Hughes  has  had  an  ac¬ 
tive  VE  program.  It  is  under  the  cogni¬ 
zance  of  the  Corporate  Director 
Product  Effectiveness.  Several  other 
management  disciplines  are  also  under 
this  corporate  office,  including  Quality, 
Reliability,  Maintainability,  Safety, 

Data  and  Configuration  Management  and 
Design  Standards.  While  specific  activi¬ 
ties  differ  among  these  programs  there 
are  many  common  objectives  and  similar 
management  principles  involved.  Grouping 
them  together  under  a  single  Corporate 
office  provides  good  visibility  and  a 
streamlined  nonredundant  approach  through¬ 
out  the  mauiy  organizational  elements  of 
the  Company. 

The  VE  program  in  brief  summary  involves 
workshop  training  sessions,  individual 
and  task  team  efforts,  and  generation 


of  lower  cost  alternatives  on  company 
products  and  supporting  activities.  There 
are  two  distinct  areas  of  work: 

(1)  On  contracts  which  have  formal  DOD 

Value  Engineering  incentives  in  accor¬ 
dance  with  the  Armed  Services  Procurement 
Regulation  1-17.  Savings  on  approved 

contract  changes  are  split  between  the 
customer  and  Hughes  using  the  incentive 
share  ratios  negotiated  in  the  contract. 
Hughes  now  has  over  100  contracts  in 
force  with  VE  incentive  clauses. 

(2)  On  other  programs  or  company  activi¬ 
ties  where  changes  do  not  require  cus¬ 
tomer  approval  as  in  (l)  above.  VE  task 
teams  are  organized  to  investigate  high- 
cost  areas  of  the  business  structure, 
frequently  during  the  course  of  new 
product  development  and  in  the  formal 
proposal  phases. 

Table  1,  "Hughes  Aircraft  Company  Value 
Engineering  Performance  I96I*  -  1979", 
presents  some  overall  statistics  on  the 
program.  Formal  contract  changes  as 
described  in  category  (l)  above  were 
incorporated  which  have  cut  the  cost  of 
contracts  by  $373  million  over  the  period. 
This  has  been  shared  in  the  amount  of 
$258  million  to  Hughes  customers  and  $115 
million  to  the  Company.  The  many  hun¬ 
dreds  of  other  cost  reduction  changes  in 
category  (2)  above  have  amounted  to  an 
additional  $200  million.  These  cost 
savings  are  about  2.3  percent  of  gross 
sales  over  the  period  —  a  very  important 
ingredient  of  Company  earnings.  The  cus¬ 
tomer  return  on  investment  is  11  to  1. 

VALUE-QUALITY  ECONOMICS 

It  can  be  argued  that  value  engineering 
changes,  considered  only  from  their  ec- 
nomic  effect  enhance  quality.  The  def¬ 
inition  of  quality  quoted  earlier  in¬ 
cludes  the  phrase,  "at  the  most  economic 
level  that  will  allow  full  customer  satis¬ 
faction".  Clearly,  quality  is  improved 
if  the  cost  is  reduced,  all  other  factors 
remaining  unchanged.  But  most  value 
engineering  changes  that  are  well  con¬ 
ceived  also  provide  other  advantages  as 
well.  Table  2,  "Value  Engineering 
Changes  That  Improved  Quality",  lists  a 
number  of  typical  changes  on  Hughes  pro¬ 
grams.  Each  of  these  has  provided  a 
beneficial  quality  effect  as  well  as  a 


cost  reduction.  These  benefits  include 
improved  reliability,  simplification, 
easier  maintenance,  improved  performance 
characteristics,  reduced  inspection  time, 
and  manufacturing  process  improvement. 

For  example,  numbers  235,  2l*2,  UU8,  55, 

3  and  2  all  improved  reliability  and 
maintainability  through  simplification 
of  design  and  reduction  of  number  of 
components.  Numbers 736  and  lU6  improved 
performance  characteristics.  Numbers  11? 
and  l8l  reduced  inspection  time  and  pro¬ 
vided  a  standard  part  in  place  of  a 
special  device,  with  resultant  logistics 
benefit . 

These  additional  benefits  seem  to  charac¬ 
terize  most  VE  projects.  Table  3,  "VE 
Fringe  Effects" ,  was  prepared  from  de¬ 
tailed  field  analysis  of  several  hundred 
VE  changes  incorporated  into  products. 

The  study  was  made  by  a  task  group  of  the 
American  Ordnance  Association  (now  re¬ 
named  American  Defense  Preparedness 
Association).  Producibllity  and  lead 
time  were  improved  in  90  and  76  percent 
of  the  cases  studied.  Reliability  and 
maintainability  were  improved  Ult  and 
1+0  percent  of  the  time.  Considering  all 
cases  the  largest  disadvantage  found  was 
only  3  percent  for  logistic  effects.  The 
disadvantages  combined  are  so  minimal 
as  to  be  negligible  in  comparison  with  the 
benefits. 

CONCLUSION 

The  value  engineering  approach  to  cost 
reduction  through  critical  examination 
of  functions  and  development  of  lower 
cost  alternatives  is  a  proven  management 
method  that  can  produce  dramatic  savings 
as  well  as  improved  product  characteris¬ 
tics.  The  combined  effect  can  provide 
a  powerful  stimulus  toward  achieving  an 
economic  quality  level  consistent  with 
full  customer  satisfaction. 


TABLE  1  -  HUGHES  AIRCRAFT  COMPANY 
VALUE  ENGINEERING  PERFORMANCE 

I96U  -  1979 


583 

VECPs  accepted  by  Hughes  customers  since  ASPR 
VE  contract  clauses  adopted  in  196^* 

U3 

Programs  on  which  VECPs  were  generated 

$373M 

Total  savings  negotiated  through  formal 

VE  changes 

Of  sales  during  period 

$258M 

Customer  Share 

$115M 

Hughes  Share 

$23. 9M 

Overhead  funds  invested  to  generate  VECPS 

11  to  1 

Customer  return  on  overhead  investment 

Program 


TABLE  2  -  VALUE  ENGINEERING  CHANGES  THAT  IMPROVED  QUALITY 
VECP 

Nximber  Description 


F-15  Radar 

235 

2U2 

TOW  Missile 

736 

117 

658 

UU8 

518 

Maverick  Missile 

77 

lk6 

198 

181 

55 

i*07L  Ops.  Center 

135 

138 

Sheridan  Laser  RF 

k 

3 

AN/SPS-52  Radar 

2 

Combine  RF  modules 

Simplify  switch  regulator  design 

Change  plastic  reflector  to  aluminum 

Replace  meter  lamp  with  standard  bulb 

Reduce  pyro  switch  sample  size 

Redesign  electrical  unit 

Modify  control  surface  finish 

Increase  shipping  container  weight 

Increase  vidicon  illumination  level 

Allow  variables  and  line  flow  sampling 

Change  connector  to  MIL-C-83T23 

Modify  missile  power  supply /relocate  circuitry 

Reduce  number  of  terminal  load  assemblies 

Simplify  design  of  HV  power  supply 

Change  dip  brazing  to  casting 

Replace  assembly  with  one  molded  part 

Simplify  antenna  design  and  construction 


WHAT  IS  MISSION  SUCCESS? 

By:  Vernon  E.  Woodln 
Quality  Assurance 
Martin  Marietta  Aerospace 
Denver  Division 

I.  INTBODUCTION 


Mission  Success  Is  our  way  of  doing 
business  (Vlewgraph  #1} .  When  our 
management  discusses  Mission  Success, 
they  are  referring  to  the  ability  of  our 
products  to  perform  successfully  In  the 
Intended  mission.  Martin  Marietta  has 
several  product  performance  histories 
that  demonstrate  the  Mission  Success 
concept.  Our  successful  performance  Is 
a  matter  of  record  and  as  a  result, 
motivates  our  personnel  to  the  continued 
emphasis  on  Mission  Success. 

High  Quality  product  performance  Is 
determined  by  three  basic  elements  that 
provide  the  proper  focus  for  Mission 
Success  (Vlewgraph  #2) .  These  Mission 
Success  elements  are  a  combination  of  a 
System.  Discipline  and  Attitude.  The 
elements  are  Interactive  and  support  each 
other. 

The  emergence  of  the  Mission  Success 
concept  at  Martin  Marietta  began  In 
late  1965  after  the  occurrence  of  errors 
that  jeopardized  critical  schedules  and 
resulted  In  functional  systems  failures 
(Vlewgraph  #3) .  At  that  time,  Martin 
Marietta  management  directed  an  examina¬ 
tion  of  all  our  assurance  systems.  The 
examination  revealed  the  systems  were 
sound  and  functioning  according  to 
documented  procedures,  but  failures  were 
still  occurring  and  problems  were  escap¬ 
ing  our  screen.  Specific  findings  were 
Identified  which  sensitized  management  to 
make  the  necessary  changes  to  adopt  the 
elements  of  Mission  Success.  These 
findings  Included  (a)  Multiple  Data  and 
corrective  action  systems  existed  In 
Quality  Engineering  and  Reliability 
Engineering,  (b)  No  specific  function 
existed  to  Interface  problem  Identifi¬ 
cation  to  the  technically  responsible 
personnel  for  specific  pieces  of  equip¬ 
ment,  and  (c)  No  specific  disciplined 
"second  look"  by  knowledgeable  and 
responsible  personnel  existed  on  flight 
committed  hardware. 


II.  THE  ELEMENTS  OF  MISSION  SUCCESS 

A.  The  System  - 

A  program/project  environment 
demands  the  establishment  of  a 
set  of  minimum  acceptable 
standards. 

The  Martin  Marietta  system 
compiles  with  the  Quality 
Military  Specification 
(Vlewgraph  #4) .  The  system  Is 
documented  in  company  standards 
and  procedures  that  define  the 
requirements  and  establish 
minimum  standards.  However,  we 
have  Incorporated  within  our 
system  some  unique  features  such 
as  emphasis  on  Quality  require¬ 
ments  and  planning  during  early 
product  development  and 
throughout  the  fabrleatlon/test 
of  the  product.  These  Include 
documented  Quality  design 
reviews  using  checklists,  vendor 
seminars.  Special  Consideration 
Items  Drawing  (SCID) ,  Corrective 
action  that  ensures  the  problem 
cause  Is  corrected,  and  Hardware 
Revlew/Fedlgree  of  selected 
Items. 

The  design  review  checklists 
provide  a  disciplined  approach 
to  the  review  which  documents 
the  design  problems  and  provides 
a  method  of  follow-up  and  docu¬ 
mented  closure.  Our  suppliers 
and  subcontractors  are  brought 
Into  our  system  through  vendor 
contacts  and  presentation  of 
data  at  vendor  seminars.  These 
contacts  Impress  on  the  suppliers 
the  Importance  of  Mission 
Success.  We  present  Mission 
Success  data  not  only  to 
supplier  management  but  also 
to  their  engineers,  technicians 
and  Quality  personnel.  The  data 
demonstrates  where  and  how  their 
particular  product  affects  the 
final  mission.  Presenting  our 
concept  of  Mission  Success,  how 
we  solve  problems,  and  manage¬ 
ment  attitudes  provides  a  more 
uniform  and  consistent  program. 
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Also,  this  approach  brings  the 
suppliers  closer  to  the  final 
product  and  demonstrates  their 
participation  In  the  end  products. 
The  other  unique  features  are 
discussed  later. 

Problem  Example  (Vlewgraph  #  5) 

System  components  were  not  analyzed 
Initially,  and  no  requirement  In  a 
procurement  specification  was 
established  until  several  rejects 
at  the  assembly  level  were  encount¬ 
ered.  By  this  time,  all  units  had 
been  delivered.  Subsequent  analysis 
established  tolerance  which  was  the 
basis  for  screening  all  existing 
units  and  for  a  specification 
change  for  future  reprocurement. 

This  was  a  problem  that  was  prevent¬ 
able  with  disciplined  engineering 
analysis  and  design  reviews.  The 
emphasis  on  early  Mission  Success 
which  assures  minimal  acceptable 
standards  are  applied  to  prevent 
these  failure  occurrences. 

Discipline  (Vlewgraph  #  6) 

1.  A  strong  and  uniform  Insistence 
on  discipline  In  all  organiza¬ 
tions  Is  essential  to  achieve 
consistent  Mission  Success . 


2.  A  set  of  checks  and  balances 
Is  required  to  assure  that 
Mission  Success  goals  are  being 
achieved. 

Martin  Marietta  Management 
established  (Vlewgraph  #7)  a  small 
tlghtly-knlt  organization  to 
Implement  the  above  corrective 
measures.  This  organization  Is 
also  chartered  to  provide  a 
continuous  evaluation  of  those 
disciplines  which  govern  and 
constitute  the  Mission  Success 
System.  At  Martin  Marietta  we 
label  this  organization  Mission 
Success. 

The  decision  to  establish  a 
Mission  Success  organization  was 
obviously  based  upon  the  desire 
to  achieve  maximum  objectivity. 

The  philosophy  underlying  this 
decision  was  that  without  specific 
responsibility  being  assigned, 
problems  tended  to  receive  less 
than  adequate  attention.  Mission 
Success  personnel  with  overall 
program  knowledge  and  visibility 
were  able  to  establish  meaningful 
priorities  In  a  multi-functional 
environment  and  to  Interface  with 
all  organizational  and  customer 
agencies  (Vlewgraph  #  8) . 


To  correct  the  findings  discussed 
In  the  Introduction,  our  manage¬ 
ment  elected  to  (1)  combine  the 
existing  data  and  Corrective 
Action  functions  Into  one 
organization,  to  focus  on  the 
correction  of  problem  causes,  (b) 
Create  an  environment  which  Inter¬ 
faces  problems /anomalies  with 
technically  responsible  personnel 
to  assure  closeout,  (c)  Require  a 
disciplined  second  look  on  speci¬ 
fically  Identified  hardware 
(l.e.,  flight  committed),  and 
(d)  Provide  full  visibility  of 
significant  problems  and  Impacts. 
This  hardline  discipline  requires 
assessment  of  Impact,  full 
customer  visibility  and  assurance 
that  the  cause  of  problems  are 
properly  Identified  and  that  the 
corrective  action  resolves  the 
cause . 


Three  specific  areas  (Vlewgraph 
#  9)  were  established  within  Che 
Mission  Success  Organization  to 
address  Problem  Identification, 
Second  Look  and  Audit  Entitles. 

Problem  Identification  and 
Notification  -  A  Center  for 
problem  accumulation  and  co¬ 
ordination,  designed  to  Integ¬ 
rate  factory,  launch  site  and 
vendor  organization  was  required 
(Vlewgraph  #10).  It  was  also 
mandatory  that  the  several 
organizational  elements  to  be 
disciplined  to  notify  the  Center 
of  project-oriented  problems. 

This  became  the  Corrective  Action 
Control  Center  (CACC) . 


Hardware  Review  -  A  system  was 
required  which  Identified  and 
Insured  that  designated  equip¬ 
ment  received  a  "second  look"  on 
an  in-line  basis  (Vlewgraph  #11). 
A  discipline  was  established 
which  would  force  this  to  occur 
on  a  systematic  basis.  The 
data  resulting  from  the  hardware 
review  became  the  Pedigree 
documentation. 

Audit  -  The  'disciplines  which 
are  the  essence  of  Mission 
Success  were  identified  and  an 
audit  plan  developed  and  imple¬ 
mented  which  would  place  those 
systems  under  continuous  monitor¬ 
ing  (Vlewgraph  //I2). 

These  disciplines  provide  the 
necessary  checks  and  balances 
when  several  organizational 
elements  are  Involved.  It  should 
be  pointed  out  that  the 
disciplines  of  Mission  Success 
identified  above  would  be  per¬ 
formed  even  if  they  were  not 
given  an  organization  identity. 
Specific  organizational  respon¬ 
sibility  provides  the  attention, 
focus  and  objectivity  to  achieve 
overall  Mission  Success 
(Vlewgraph  #13). 

Example  Problem  -  This  problem 
demonstrates  the  depth  to  which 
our  Mission  Success  Program 
extends  to  Identify  and  correct 
problems  to  prevent  failures  in 
the  final  product.  The  evalua¬ 
tion  and  disposition  covered  a 
span  of  six  (6)  months. 

An  assembly  failed  test  in  the 
electronics  test  area.  The 
failure  cause  was  traced  to  a 
single  component  that  had  a 
fractured  solder  attachnent  lead 
to  body.  A  short  time  later,  a 
module  from  the  same  assembly 
configuration  failed  the  pre-pot 
test  and  the  same  basic  compon¬ 
ent  was  determined  to  be  the 
culprit.  A  Corrective  Action 
Problem  Siramary  was  opened  to 
identify  Impacts  against  all 
functional  end  products.  Failure 
analysis  was  started,  pull  tests 


and  twist  tests  were  performed, 
and  microsectionlng  was  accom¬ 
plished.  These  investigations 
indicated  voids  in  the  lead 
attachments  and  missing  body 
attachments.  It  was  determined 
from  the  failed  components  that 
only  certain  lots  were  at  fault. 
The  problem  lots  were  scrapped 
and  suspected  components  in 
modules/assemblies  as  determined 
from  our  traceability  system, 
were  replaced  with  good  compon¬ 
ents.  Other  corrective  and 
preventative  measures  Included 
changing  the  assembly  process 
plan  to  prohibit  the  use  of 
problem  date  codes;  recommending 
an  automatic-tracking  system 
that  statuses  part  number,  date 
code,  quantity  manufacturer  and 
purchase  order  number;  and 
changing  the  quantity  of 
components  subject  to  pull  test 
within  each  dash  number. 


C.  Attitude 


Mission  Success  demands  an 
absolute  intolerance  of  failure 
from  all  levels  of  the  organi- 
zatlon. 


Although  attitude  is  being  dis¬ 
cussed  as  the  third  element  of 
Mission  Success,  it  is  as  import¬ 
ant  and  in  some  Instances,  more 
Important  than  the  other  two 
elements  (Vlewgraph  #14).  As  I 
mentioned  early  in  this  paper, 
our  management  considers 
Mission  Success  "a  way  of  doing 
business".  Management  estab¬ 
lished  as  a  basic  philosophy 
that  no  failures  would  be 
tolerated  for  the  functional  end 
products.  Management  must 
develop  a  "success  oriented" 
attitude  and  accept  nothing  short 
of  success.  Management  must 
Supply  the  resources  to  foster 
and  nurture  the  "Mission  Success" 
attitude  so  that  the  attitude 
permeates  all  levels  of  personnel 
and  is  accepted  in  their  day-to- 
day  activities. 
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When  personnel  have  a  Mission 
Success  attitude,  the  system 
and  the  discipline  reinforce 
attitude  and  conversely, 
attitude  Is  reflected  In  the 
system  and  discipline. 

2.  Problems  should  be  aggressively 
Identified,  catalogued  and 
given  proper  visibility  to 
assure  management  attention 
and  resolution. 

Once  the  proper  attitude  toward 
success  was  established,  the 
system  was  required  to  Identify 
problems,  provide  docimentatlon, 
cataloguing  and  access  to  data, 
and  notify  the  appropriate 
personnel  for  problem  resolution. 
All  possible  data  sources  must 
be  made  available  for  evalua¬ 
tion  and  problem  analysis.  A 
problem  Identification  and 
notification  discipline  must 
be  structured  to  act  as  a 
"filter"  so  that  management  is 
not  deluged  with  problems  whose 
nature  may  be  readily  handled 
In  a  routine  manner. 

SUMMARY  - 

An  organization  must  think  Mission 

Success  to  obtain  success. 


Three  elements  are  key  to  Mission 
Success  -  System.  Discipline  and 
Attitude.  Management  must  decide 
on  and  enforce  a  policy  of  Intoler¬ 
ance  to  failures  in  the  final  pro¬ 
duct.  They  must  provide  an  environ¬ 
ment  organizationally  that  enforces 
and  promotes  this  philosophy. 

Impact  assessment  must  be  accom¬ 
plished  and  full  visibility  provided 
to  management  and  the  customer  on 
significant  problems.  The  customer 
should  be  made  a  part  of  the  Mission 
Success  concept  so  overall  program 
schedule  and  cost  Impacts  can  be 
assessed.  Usually  when  the  customer 
Is  aware  of  problems,  work-arounds 
or  alternate  solutions  are  available 
to  reduce  or  eliminate  overall 
program  cost  and  schedule  Impacts. 
The  Mission  Success  concept  Is 
equally  applied  to  the  development 


and  test  of  software  products 
and  hardware/software  system. 
Mission  Success  at  Martin 
Marietta  Is  an  essential  part 
of  doing  business. 
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WHAT  IS  MISSION  SUCCESS 


INTRODUCTION 


MISSION  SUCCESS 


yOOBLEH  {lAHPlE 

0  SEVERAL  REJECTS  AT  ASSENaiY  LEVEL 

0  SystM  Canponents  Not  AA«ly2f4 

0  CenpoMfit  Rrocuremeni  Sp«c<f<c«t<on  dtd  not  Est«ti1Uh  SisH 
for  Scrceolnq 

0  All  ItCAL  Delivered  Before  Problem  Surfaced 
0  rOMCCTlVE  MEASURES 

0  An«)yt«i  to  Identify  Screenino  ReoulreMents  for  CoMponent 
0  Cnenged  Procuremnt  Specificetlon  for  Future  Reprocurewnt 


Tbit  proble*  ae«  preventible  by  disciplined  enelytls  and  oilnimum  standards  identification. 


DISCIPLINE 


A  STRONG  UNIFONN  INSISTENCE  ON  DISCIPLINE  IN  ALL  ORGANIZATIONS  IS  ESSENTIAL 
TO  ACHIEVE  CONSISTENT  MISSION  SUCCESS. 


MISSION  SUCCESS  DEMANDS  AN  ABSOLUTE  INTOLERANCE  OF  FAILURE 
FROM  ALL  LEVELS  OF  THE  ORGANIZATION. 


0  COMBINE  the  existing  DATA  AND  CORRECI  .  ACTION 
FUNCTION  INTO  ONE  ORGANIZATION. 

0  CREATE  A  FUNCTION  TO  INTERFACE  PROBLEMS  WITH 
technically  RESPONSIBLE  PERSONNEL  AND  ASSURE 
THEIR  CLOSEOUT. 


0  REQUIRE  A  DISCIPLINED  SECOND  i.DOk  ON  FLIGHT 
COMMITTED  HARDWARE. 


INTRODUCTION 


HISTORY  -  LATE  65 

SYSTEM  FAILURES  CAUSED  AN  EXAMINATION  OF  OUR  "ASSURANCE" 
CHECK  ANO  BALANCE  SYSTEMS. 

OUTPUTS  RELATIVE  TO  MISSION  SUCCESS 

0  DATA  ANO  CORRECTIVE  SYSTEMS  EXISTED  IN  QUALITY 
ENGINEERING  AND  RELIABILITY  ENGINEERING. 

0  NO  SPECIFIC  FUNCTION  EXISTED  TO  INTERFACE  PROBLEM 
IDENTIFICATION  TO  PERSONNEL  WHO  WERE  TECHNICALLY 
RESPONSIBLE  FOR  THEM, 

0  NO  SPECIFIC  DISCIPLINED  SECOND  LOOK  BY  KNOWLEDGE¬ 
ABLE  ANO  RESPONSIBLE  PERSONNEL  EXISTED  ON  FLIGHT 
COMMITTED  HARDWARE. 


oiscipnwt 


4  iff  Of  Ct*lCK5  AM>  iAlAUCt^  ;5  WOUJBfP  TO  ASSUBf  T>«T  MJSSJON  SUCCESS 
GOALS  APE  BEING  AChICVED. 

QPGAMEATIOW  StLtCTlQM 

MANAOCHENT  OCVElOPED  A  SHALL.  TIGhTIT  KNIT  ORGANI7ATION  TO 
IMPLCMCNI: 


0  Mi$Mon  Success 

0  Sin9leness  of  Objective 

0  Concentrated  Effort 


COBRECTIVI 

ACTION 


TMt  -SrSTEM" 


ORAHINGS 

PROCESS  PLANS 

LAUNCH  FACILITT 

ANALYSIS 

PRaCOURES 

MISSION  OPS. 

PROGRAM  PLANS 
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INTERFACE  SPECS. 
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FLIGHT  READINESS 

CONTRACT  END  (TENS 
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ACCEPTANCE 

CORL 

ENVlROMtENTAL  DEFINITION 

HORSE  CASE  ANALYSIS 

PHYSICAL  CONflGURATION 

NISSION  SOFTMRE  programs 

AEROOVMMtC  (AIR  LOADS) 

AUDIT 

theiwal  control 

Thermal 

Pc  PART  DERATING 

ENVIROmCNTAL  ACCEPT- 

TRAJECTORY 

ACOUSTIC 

ANCE  TEST 

DISCRETES 

VIBRATION 
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PEDIGREE 

SHOCK 

Pc  PARTS 

Flight  riaoiness 

STATIC  1  DYNAMIC  LOADS 

CIRCUITS 

ACRBC  MO  SC  ID 

flECTR9IACNCT|C  INTERFACE 

ELEC. 

Flight  anomaly  mtgs. 

HECH. 

TRAINING  ANO  CERTIFI- 

MARGINS  OF  SAFfTr 

CatPONENTS 

CATION 

CAPS  NlfTINCS 

IVWDHARE 

SUSSYSTCNS 

PERFORMANCE 

SYSTEMS 

-ONRECTIVE  ACTION 

qualification  I  reliability 

FAILURE  analysis 

TESTING 

OEVCLOFMINT  TESTS 

PAnOAO  INTEGNITV 

NISSIOH  SUCCESS 

INTERFACE  INTEGRITY 

PRODUCT  INTEGRITY 

SUSPECT  MATERIAL 

ENGINEER 

REPORTS  AND  GIOEP 

tritatttffr.  MidraiMtHMY.  numnr  {NSHKCtinc.  sAfcrr,  cAfv  safitt.  ouaiitv  assmance,  cont.  coNteoi, 
TtAlHtHG  KUD  C(tTIl|CATION.  AUDIT,  SPECIAL  STSTCM  RIVIfIfS.  TRENDS.  ANOMALIES  VS.  FAILURfS.  TERMS. 
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CORRECTIVE  ACTION  INTERFACES 


CORBtCTIVE  ACTION  CONTROL  CENTER 


f  iNAl 
ASSEMDl* 


CUSTOMER 

REPRESENTATIVE 


DISCIPLINE 


P'SClPLIMt 


AREAS  OF  RESPONSIBILITY: 

PROBLEM  IDENTIFICATION  AND  NOTIFICATION 
0  CACC 

DISCIPLINES  SECOND  LOOK 
0  HARDWARE  REVIEW 

CONTINOUS  MONITOR  OF  MISSION  SUCCESS  DISCIPLINES 
0  AUDIT 

Cl- .  :pl!s( _ 


"‘■CBlf*  rOIH’lfKATJOM  *_MC  MOT|FH.AT|ON 


OISCIPLINE 

HARDWARE  REVIEW  PROCESS: 

0  RE-EXAMINATION  Qf  ALL  DATA  ASSOCIATED  WITH  THE  VEHICLE 

0  REVIEWS  ARE  CONDUCTED  AT  SELECTED  MILESTONES  IN  THE  VEHICLE 
SEQUENCE  BY  INOEPENOENT  PERSONNEL 

0  PROVIDES  CERTIFICATION  Of  FLIGHT  WORTHINESS 
TOOLS  USED  TO  ACCOMPLISH  REVIEW  PROCESS- 

0  IDENTIFICATION  OF  EQUIPMENTS  SUBJECT  TO  HARDWARE  REVIEW 
SC  ID 

0  performance  of  review  and  REPORT  PREPARATION 
COMPONENT  AND  SYSTEM  SUMMARIES 

0  SYSTEM  IS  DISCIPLINED  BY  EXERCISING  CONTROL  OF  HARDWARE 
COMPONENT  CERTIFICATE 
INCREMENTAL  SlMURY  REVIEW 


DISCIPLINE _ 

AUDIT  PROCESS: 

METHOD  OF  INDEPENDENTLY  ASSESSING  DISCIPLINES  INTO  HARDWARE  AND  SYSTEMS 

0  SUPPLIER  SURVEILLANCE 

0  HARDWARE  AUDIT 

0  FACILITY  AUDIT 

0  SYSTEM  AUDIT 

0  VERIFICATION  AUDITS 

0  SPECIAL 


MISSION  SUCCESS  DOCUMENTATION: 

CORRECTIVE  ACTION  PROBLEM  SUMMARY  REPORTS 
COMPONENT  PEDIGREES 
INCREMENTAL  SUMMARY  REVIEW  HISTORIES 
PROGRAM  STATUS  REPORT 
PROGRAM  AUDIT  REPORTS 

REVIEWS: 

WEEKLY  MEETING  TO  DISCUSS  MAJOR  IMPACT  PROBLEMS 
SEMI-ANNUAL  PRXRAM  REVIEWS 
DESIGN  REVIEWS 
READINESS  REVIEWS 
BUILD  HISTORY 

PROBLEMS  ENCOUNTERED  DURING  TEST  AND  CORRECTIVE  ACTION 

MISSION  OBJECTIVES 

MARGINS 

QUALIFICATION  STATUS 


■nSION  SlWlFSS  UWNCIS  ««■  IBSOlUtI  IWtCHlIiAllCt  OF  rtUuBC  'BW  III  LF^flS  OF  T«f 


S  U  W  w  *  B  * 


0  A  BBXBMI/BBOJICT  {NyiBDWFfNT  OCUhOS  THE  CSTAM.t$)«*(liT  QF  A  SET  OF  MINIMUM 
ACCCPTAeiE  STAMOAanS- 

0  A  SIBONG  AND  UNIFOBM  INSISTANCE  ON  DTSCIPlINI  IN  AU  OBGANIEATIONS  IS  ESSENTIAL 
TO  ACHIEVE  COMSISTENT  MISSION  SUCCESS 

0  A  SET  Of  CHECKS  AND  lAiANCES  IS  REQUIRED  TO  ASSURE  TmAT  MISSION  SUCCESS  GOAiS 
ABE  being  achieved. 

0  HISS:ON  SUCCESS  DEMANDS  Ah  ABSOlUTE  INTQlERAPtCE  0*  FAILURE  rR|><  All  LEVEiS  OF 
THE  ORGANISATION 

0  MOBiCMS  Should  be  aggressively  identified,  catalxued  and  given  proper  visiiuitv 

TO  ASSURE  MANAGEMENT  ATTENTION  AND  RESOlUTIOK. 
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HEAO  PROTOFLIGHT  APPROACH 

James  H.  Robinson 
MSFC 

Assessment  of  the  HEAO  Protoflight  Approach 

PREFACE 

The  HEAO  Project  employed  many 
special  features  that  were  enacted  by 
initial  contract  to  establish  a  low  cost 
program  which  would  be  technically  ade¬ 
quate  and  within  an  acceptable  risk  range . 
Possibly  the  most  significant  of  the 
program  features  is  the  protoflight 
concept . 

The  protoflight  concept  features 
building  a  single  set  of  hardware  which 
serves  the  purposes  of  both  the  ground 
test  model  and  the  flight  model.  By 
making  maximum  use  of  previously  space 
qualified  components  developed  for  other 
programs,  the  actual  number  of  test  oper¬ 
ations  could  be  held  to  a  minimum  with 
most  of  the  testing  being  done  at  higher 
assembly  levels  of  hardware.  Analysis 
played  an  important  role  in  many  of  the 
well  established  engineering  disciplines 
since  it  served  as  the  only  method  of 
verification  in  many  of  the  less  critical 
applications  or  in  areas  where  large 
design  safety  factors  could  be  used. 

Space  technology  maturity  was  also  a 
main  factor  in  the  decision  to  employ 
the  protoflight  concept. 

The  experience  gained  in  HEAO  flight 
operations  sufficient  to  assess  the 
protoflight  approach  as  a  viable  method 
of  qualifying  space  flight  hardware . 

This  document  evaluates  the  HEAO  proto¬ 
flight  approach  and  shows  the  time  and 
cost  savings  as  well  as  some  disadvantages 
of  not  having  operational  test  hardware. 

INTRODUCTION  AND  SUMMARY 

1.  HEAO  Description 

The  HEAO  Project  involves  three 
orbital  payload  mission  launches  from 
KSC  by  the  Atlas  Centaur  launch  vehicle 
at  Intervals  of  approximately  one  year. 
Each  payload  consists  of  an  observatory 
system  made  up  of  two  sections.  One 
section  is  the  spacecraft  equipment  mod¬ 
ule  (SEM)  containing  all  the  ancillary 
equipment  (subsystems)  coimnon  to  all 
three  missions.  The  other  section  is 
the  experiment  module  (EM)  containing  a 
complement  of  mission  peculiar 
experiments . 


The  overall  physical  dimensions  of 
the  observatories  vary  from  14  ft.  to  22 
ft.  long  and  a  diameter  of  approximately 
8  ft.  The  weights  vary  from  6500  lbs.  to 
7000  lbs.  The  basic  observatory  designs 
were  completed  in  1974. 

The  basic  overall  science  objectives 
of  the  HEAO  are  as  follows: 

Map  the  sky  for  sources  and  back¬ 
ground  of  X-rays . 

Locate  and  examine  in  detail  individ¬ 
ual  X-ray  sources  and  determine  intensity, 
energy  spectra,  and  temporal  behavior. 

Determine  the  spaclal  structure  of 
extended  X-ray  sources . 

Determine  intensity,  energy  spectra 
and  temporal  behavior  of  gamma-ray  flux. 

Measure  energy  spectra  composition 
and  flux  of  cosmic  rays . 

The  specific  role  each  observatory  plays 
in  meeting  the  objectives  of  the  overall 
program  is  as  follows : 

HEAO-1  was  a  scanning  mission  which 
made  a  complete  survey  of  the  celestial 
sphere  (all  sky)  for  discovery  and 
location  of  X-ray  sources.  Also,  some 
pointings  at  specific  sources  were 
accomplished , 

HEAO-2  is  a  high  resolution  pointing 
mission  for  studying  specific  X-ray 
sources  in  detail. 

HEAO-3  is  a  scanning  mission  which 
makes  a  complete  survey  for  gSmma-ray' 
and  cosmic  radiation. 

2.  Conventional  Method 

The  conventional  concept  for  devel¬ 
oping  orbiting  payloads  featured  building 
two  Ideally  identical  articles.  The 
first  article,  termed  the  prototype,  was 
test  dedicated  and  preceded  the  second 
article  long  enough  to  incorporate  the 
knowledge  and  experience  gained  into  the 
second  article  which  was  the  scheduled 
flight  article. 

3.  HEAO  Protoflight  Approach 

The  HEAO  protoflight  approach  con¬ 
sisted  of  a  program  formulated  by  the 
contractors  in  1974  from  NASA  guidelines 
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and  requirements  to  develop  flight  hard¬ 
ware  by  using  a  combination  of  test  and 
assessments  for  operational  qualification 
involving  a  minimum  of  dead-end  (test 
dedicated)  hardware.  Reliability  trades 
were  conducted  to  select  the  most  cost 
effective  combination  and  still  maintain 
an  acceptable  probability  of  success. 

The  minimum  verification  requirements 
were  defined  in  the  beginning  and  placed 
on  the  initial  contract.  These  require¬ 
ments  were  identical  for  all  experiments. 
The  spacecraft  and  observatoryi^^require- 
HKnts  were  different  from  the  single 
mission  experiment  requirements  since  the 
observatory  requirements  contained  the 
integration  provisions  and  varied 
slightly  to  accommodate  the  multiple 
mission  applications.  The  protoflight 
concept  was  used  for  all  three  observa¬ 
tories  . 

In  order  to  make  the  protoflight  approach 
effective,  the  initial  design  work  gave 
much  consideration  to  structural  strength 
and  operational  life.  Also,  criteria 
were  predicted  on  the  assumption  that  all 
electronic  piece-parts  would  be  screened 
prior  to  fabrication^^) .  All  environments 
of  testing,  transportation,  handling, 
launch  and  operations  were  considered. 
Redundancy  was  employed  in  critical  appli¬ 
cations  to  assure  required  reliability. 

A  testing  program  was  devised  in  which 
all  potentially  destructive  testing  was 
carefully  controlled  with  the  environ¬ 
mental  levels  only  slightly  higher  than 
those  expected  during  the  hardware  life. 
Generous  structural  and  operational  mar¬ 
gins  were  used  in  the  design.  Test 
durations  at  the  elevated  environmental 
levels  were  reduced  to  preserve  the 
lifetime  of  the  hardware. 


(1)  The  spacecraft  is  defined  as  the 
supporting  structure  and  the  operational 
subsystems;  the  observatory  is  the  space¬ 
craft  with  the  experiments  installed. 

(2)  HRA-2001  Assessment  of  the  HEAO 
Central  Parts  Control  (CPC)  Aug.  1,  1978. 


4.  Significant  Findings 

The  significant  findings  based  on 
actual  experience  with  the  HEAO  proto - 
flight  approach  are  as  follows : 

a.  The  hardware  was  qualified 
for  space  operations . 

b.  The  reliability  was  adequate. 

c.  Significant  cost  savings 
were  realized. 

d.  Significant  schedule  savings 
were  realized. 

e.  Opportunity  for  off-line 
troubleshooting  was  limited 
since  test  hardware  was  not 
available . 

f.  Cannibalization  of  test 
hardware  to  replace  failed 
flight  hardware  was  limited. 


BASIC  APPROACH 
1.  General 

Each  contractor  planned  his  respec¬ 
tive  verification  program  within  the 
guidelines  provided  in  the  basic  contract. 
Trade  studies  were  conducted  to  evaluate 
how  much  hardware  testing  could  be  elim¬ 
inated  by  verification  through  assessment 
and  to  identify  the  hardware  level  at 
which  testing  would  give  the  most  desired 
results  (e.g.,  experiment  contractor  con¬ 
ducting  the  test  at  experiment  level  or 
deferring  the  test  to  be  conducted  at  the 
observatory  level).  These  studies  also 
established  criteria  for  conducting  the 
necessary  tests.  Resulting  requirements 
were  coordinated  with  the  other  contrac¬ 
tors  and  also  judged  against  the  interface 
control  requirements  and  were  eventually 
approved  by  MSFC  as  part  of  the  end  item 
specifications . 

The  contractors  then  prepared  control 
documentation  in  the  form  of  test  specifi¬ 
cations  and  procedures.  These  were 
reviewed  by  MSFC  and  other  contractors, 
and  approved  for  conduct  of  tests.  All 


testing  was  the  responsibility  of  the 
contractors  and  was  done  either  at  their 
own  facilities  or  at  leased  facilities. 

The  testing  was  done  under  the  surveil¬ 
lance  of  Government  quality  representatives. 

Although  some  specific  variations  existed, 
the  basic  approach  for  the  experimenters 
was  to  subject  the  experiment  as  a  unit 
to  the  required  environments .  Minimizing 
environmental  testing  at  the  component 
level  drastically  reduced  the  number  of 
total  tests  performed.  Since  the  space¬ 
craft  is  more  critical  to  mission  success 
than  an  individual  experiment,  a  selected 
combination  of  component  environmental 
and  system  level  testing  was  Implemented 
on  the  spacecraft. 

2 .  Requirements 

A  basic  set  of  common  requirements 
was  placed  on  contract  and  adhered  to  for 
all  three  missions.  Any  deviation  from 
those  requirements  entailed  action  by  the 
contracting  officer.  Those  requirements 
are  summarized  as  follows; 

a.  Use  proven  designs,  off-the- 
shelf  hardware,  design  tech¬ 
niques  and  emphasis  on 
assessment  to  minimize  the 
need  for  testing/retesting. 

All  off-the-shelf  hardware 
and  designs  were  analyzed 
prior  to  acceptance  for  use . 

b.  Identify  lower  than  system 
level  (parts  and  components) 
hardware  by  study/ ana lysis 
which  requires  special 
development  and  qualification 
testing. 

c .  Perform  in-process  acceptance 
and  selected  environmental 
testing  of  flight  hardware 

at  lower  than  system  level 
to  reduce  the  risk  of  failure 
at  systel  level  test. 

d.  Perform  qualification/accept¬ 
ance  testing  of  flight  hardware 
(protoflight  approach).  Note: 
the  experiments  were  qualifi¬ 
cation/acceptance  tested  at  the 
experiment  level  prior  to 
delivery  to  the  observatory 


contractor.  After  Integration 
into  the  observatory,  the 
system  was  subjected  to  observ¬ 
atory  system  qualification/ 
acceptance  testing. 

e.  Perform  test  types  with 
environmental  exposures  as 
shown  an  Table  I  for  experi¬ 
ments  and  Table  II  for  the 
observatory.  The  hardware 
flow  is  shown  on  Figure  1. 

3.  Types  of  Tests 

a.  Development;  Development  tests 
were  conducted  on  lower  level 
hardware  at  the  contractors' 
discretion  to  verify  the  feasi¬ 
bility  of  new  design  approaches. 
This  was  particularly  true  for 
experiments  since  the  design  of 
many  experiment  conq>onents  was 
pushing  the  state-of-the-art. 
Some  simple  balloon  flights 
were  conducted  on  selected 
experiment  components  in  order 
to  verify  the  instrument 
scientific  capability.  All 

of  these  served  as  confidence 
builders  that  the  final  product 
would  perform  satisfactorily. 

b.  In-Process  Acceptance:  Hardware 
in-process  verification  (func¬ 
tional  tests  and  inspections) 
was  performed  on  lower  levels 
(piece -parts,  components,  sub- 
assemblies,  etc.)  to  establish 
compliance  with  drawings  and 
specifications .  As  the  experi¬ 
ments  and  spacecraft  were 
assembled  from  lower. level 
hardware,  in-process  functional 
tests  and  inspections  were 
performed  to  verify  each 
installation.  At  completion 

of  assembly,  a  system  functional 
checkout  was  performed. 

*c.  Qualif Ication/Acceptance  (Q/A): 
The  end  item  flight  hardware 
was  subjected  to  a  combination 
of  qualification  and  acceptance 
verification  performed  at  the 
component  or  system  level.  The 
tests  were  used  to  demonstrate 
that  the  hardware  met  all  of  the 


4. 


Test  Sequence 


perfoinnance  and  design  require¬ 
ments  under  anticipated  opera¬ 
tional  regimes  and  environments 
defined  In  the  end  Item  speci¬ 
fication.  While  Q/A  tests 
were  performed  principally  on 
the  end  Items,  selected  com¬ 
ponents  used  In  critical 
applications  on  the  experiments 
were  subjected  to  Q/A  testing. 
On  the  spacecraft,  forty-eight 
components  were  tested  for 
qualification  and  then  refur¬ 
bished  as  required  and  reserved 
as  flight  hardware  spares . 

*Since  only  one  set  of  each  kind 
of  hardware  was  manufactured, 
the  Q/A  test  levels  were  adjusted 
to  assure  that  the  hardware  was 
capable  of  withstanding  antici¬ 
pated  operational  environmental 
levels  but  not  tested  to  fatiguing 
levels  to  determine  design  margins. 
Generous  safety  factors  based  on 
analysis  were  used  in  the  designs. 
The  following  data  will  give 
Insight  to  the  approach: 

Design  Safety  Factors 
Fluid  lines,  fittings, 
functional  components  .  .  . 
test  2  X  operating,  ultimate 
4  X  operating 

Pressure  vessels  . . .  test 
1.5  X  operating,  ultimate 
4  X  operating 

Structures  (no  static  test)... 
design  yield  2.0,  design 
ultimate  3.0 

Vibration  (sine  and  random)... 
tested  1  min.  each  axis  at 
upper  Itr  fit.  level 

Acoustic. . .tested  1  min.  at 
Ir  upper  fit.  level 

Thermal .. .tested  to  design 
operation  max  +  (5  to  10°F) 
and  min.  -(5  to  10°F) 

Electrical  Parts .. .derated  to 
0.2  to  0.75  (avg.  0.5)  of 
design  value 


Figure  2  Is  a  composite  chart 
showing  the  sequence  In  which  the  HEAO 
was  verified  beginning  with  the  system/ 
experiment  level.  Experiment  baseline 
functional  tests  and  final  acceptance 
functional  tests  embody  "on  orbit"  type 
commands  of  all  planned  modes.  Other 
functional  tests  were  designed  to 
demonstrate  experiment  Integrity  after 
environmental  exposure . 

After  completion  of  each  verification 
sequence  (experiment,  spacecraft,  observ¬ 
atory),  a  joint  contractor/MSFC  review 
of  the  qualification  status  was  made  to 
evaluate  the  readiness  to  go  to  the  next 
phase.  The  results  were  recorded  in  an 
endorsement  to  the  Certificate  of  Flight 
Worthiness  (COFW) .  In  some  cases,  the 
experimenters  were  able  to  defer  some 
tests  until  the  experiment  was  Integrated 
into  the  observatory.  Then  the  test  was 
conducted  at  the  observatory  level  (e.g., 
HEAO-2  experiment  vibration  and  acoustic 
tests  and  the  HEAO-1  A-1  experiment 
fluid  leak  tests)  .  These  actions  further 
reduced  the  overall  number  of  separate 
tests,  thereby  effecting  a  cost  saving. 
These  cases  were  evaluated  on  an 
individual  basis  and  recorded  in  the 
COFW. 

ASSESSMENT 

1.  Reliability  and  Performance 

A  definition  of  mission  success 
was  established  In  advance  of  launch. 
Flight  performance  then  could  be  measured 
against  that  success  definition  to  deter¬ 
mine  the  effectiveness  of  the  mission. 
While  Inflight  failures  and  problems 
occurred  (34  incidents  in  the  first  15 
months  of  operations)  on  HEAO-1,  these 
were  either  In  non-critical  applications 
or  In  redundant  applications  which 
resulted  at  most  In  reduced  sensitivity 
or  altered  the  calibration  in  some  of 
the  experiments.  The  design  life  of  the 
observatory  was  six  months.  After  15 
months,  all  4  experiments  were  still 
acquiring  good  and  useful  data  although 
the  mission  was  changed  from  a  "scanning 
only"  mission  to  a  more  complex  pointing 


mission  after  the  observatory  was  in 
orbit.  No  incidents  or  operational 
characteristics  that  would  justify  a 
prototype  observatory  are  evident. 

In  the  conventional  concept ,  the  tests 
for  the  prototype  are  designed  to  find 
design  deficiencies,  and  the  tests  for 
the  flight  article  are  designed  to  find 
workmanship  defects.  In  the  protoflight 
concept,  the  tests  are  designed  to  find 
both.  Since  neither  design  nor  manufac¬ 
ture  is  a  perfect  science,  some  inherent 
differences  will  exist  between  the  pro¬ 
totype  and  the  flight  models.  These 
differences,  even  when  small,  can  cause 
concern  over  whether  the  flight  model  is 
representative  enough  to  withstand  the 
natural  and  Induced  environments  based 
on  testing  the  prototype  to  specified 
levels  alone.  Confidence  in  the  proto- 
flight  is  high,  since  it  has  been  sub¬ 
jected  to  the  actual  environments. 

Further,  there  is  no  evidence  that  the 
ground  testing  created  any  defects  in  the 
protoflight  model.  In  the  experience  on 
HEAD,  no  compelling  evidence  manifested 
itself  in  favor  of  having  a  prototype. 

2 .  Manufacturing  and  Schedule 

Figure  3  shows  a  sample  manufac¬ 
turing  and  test  schedule  for  a  typical 
HEAO  observatory  based  on  initial  planning 
with  the  actual  manufacturing  and  testing 
factored  in.  Non-recurring  tasks  such  as 
definition,  design,  launch  and  mission 
operations  are  not  shown  since  those 
items  are  common  for  both  conventional 
and  protoflight  approaches.  For  purposes 
of  this  report,  the  hypothetical  proto¬ 
type  observatory  would  be  assembled  using 
the  prototype  experiments.  All  hardware 
would  be  manufactured  identical  to  the 
flight  article  except  commercial  electri¬ 
cal  piece -parts  would  be  permitted  if 
preferred  screened  parts  are  not  avail¬ 
able  .  As  can  be  seen  from  Figure  3 ,  a 
schedule  time  saving  of  approximately 
one  year  was  realized.  This  can  be 
interpreted  as  an  extra  year's  use  of 
manufacturing  and  test  facilities,  main¬ 
taining  the  full  con^lement  of  manpower 
for  purchasing,  contracting,  program 
control,  manufacturing,  testing  and 
management  as  well  as  the  cost  of  pur¬ 
chasing  parts,  materials,  components ,etc. 
for  the  extra  observatory  required  for 


the  conventional  approach.  The  time' 
saving  shown  in  Figure  3  is  considered 
an  average  for  the  protoflight  approach 
over  the  conventional  approach  where  a 
single  set  of  support  equipment  (SE) 
would  be  used  to  perform  all  testing.  A 
second  set  of  SE  would  allow  some  over¬ 
lap  in  testing  during  the  Integration 
and  Test  (I&T)  period,  but  the  saving 
in  time  would  not  be  significant  since 
test  evaluation  of  each  environmental 
test  on  the  prototype  model  would  be 
needed  for  designing  the  test  for  the 
flight  hardware.  Also,  extra  personnel 
would  be  required  to  conduct  the 
parallel  testing. 

Although  HEAO-A  was  manufactured,  tested 
and  delivered  to  the  launch  site  on  the 
original  schedule,  launch  was  delayed 
almost  four  months  due  to  a  faulty  Rate 
Gyroscope  Assembly  (RGA) . 

The  natural  question  which  arose  was 
"would  that  delay  have  happened  if  a 
conventional  approach  had  been  used?" 

This  question  cannot  be  answered 
directly.  The  RGA  electronics  was  a 
new  design  being  developed  for  another 
space  project  which  was  scheduled  to 
precede  HEAO-A  by  some  12  months .  That 
project  fell  behind  with  the  attending 
delay  in  need  for  RGA's.  This  positioned 
HEAO  in  line  for  Initiating  the  RGA 
development  for  space  use. 

The  RGA  purchase  order  for  three  HEAO 
missions  was  one  Engineering  Model  (EM) 
and  eight  flight  units .  The  first  unit 
was  scheduled  to  be  used  for  qualifica¬ 
tion  tests  and  later  refurbished  for  a 
flight  spare.  The  manufacturer's 
delivery  was  several  months  behind 
schedule.  Consequently,  there  was  not 
enough  time  between  the  EM  testing  and 
flight  hardware  production  to  provide 
full  benefit  from  the  EM  tests.  Even 
under  the  above  conditions,  a  significant 
problem  emerged  only  about  two  weeks 
prior  to  scheduled  launch  and  then  only 
through  extended  operational  tests. 

With  the  mission  criticality  of  the  RGAs 
being  very  high,  the  decision  was  made 
to  hold  the  launch  until  the  generic 
problems  were  understood  and  fixed, 
although  a  working  flight  spare  was 
available .  An  observatory  was  not 
needed  to  perform  the  requalification 


testing  as  the  offline  testing  was  suc¬ 
cessful.  Prototype  observatory  testing 
may  not  have  uncovered  the  seriousness 
of  the  problem  since  it  would  not  have 
been  scheduled  for  prelaunch  testing, 
the  testing  on  which  the  problem  was 
discovered.  In  that  event,  the  existence 
of  a  prototype  would  not  have  enhanced 
the  launch  schedule .  The  projected 
launch  schedule  for  a  conventional 
development  approach  would  have  been  one 
year  later  than  the  protoflight  approach. 
Assuming  that  the  problem  was  found  in 
prototype  observatory  testing  and  solved 
without  affecting  the  launch  schedule, 
the  protoflight  launch  was  still  eight 
months  earlier  than  it  would  have  been 
with  the  conventional  approach. 

Some  disadvantages  of  not  having  a  proto¬ 
type  exist.  In  the  event  of  a  catastro¬ 
phic  accident  such  as  may  occur  in 
transportation  or  handling  or  failure  to 
achieve  proper  orbit,  a  prototype  could 
be  refurbished  and  launched.  In  the 
event  of  subsystem  or  component  failure, 
prototype  hardware  could  be  cannibalized 
for  the  flight  vehicle  provided  that  the 
prototype  is  of  flight  quality.  Diag¬ 
nostics  for  inflight  anomalies  could  be 
conducted  on  the  ground  using  the  proto¬ 
type. 

3.  Cost  Evaluation 

A  study  was  conducted  based  on 
available  data  to  determine  what  cost 
savings  were  realized  by  using  the  proto- 
flight  concept  in  lieu  of  the  conventional 
concept.  A  ground-rule  was  established 
whereby  costing  covered  all  aspects 
leading  up  to  having  an  observatory 
ready  for  launch  operations.  Costing 
was  put  into  three  main  categories ;  (1) 

Design  and  development,  (2)  Manufacturing 
and  (3)  Integration  and  test.  For  pur¬ 
poses  of  this  study,  the  conventional 
approach  would  designate  the  protoflight 
article  as  the  prototype  article  and  a 
flight  article  would  be  added.  Then, 
the  cost  differential  would  be  the  manuf 
facture,  integration  and  test  of  the 
flight  article.  In  assessing  the  cost 
differential  between  the  two  approaches, 
it  was  assumed  that  due  to  the  hardware 
manufacturing  learning  curve,  reuse  of 
certain  fixtures,  etc.,  the  second 


article  would  cost  10%  less  to  manufac¬ 
ture  than  the  first  article.  Further, 
applying  the  same  principles  to  inte¬ 
gration  and  test,  a  207o  reduction  in 
cost  was  assumed.  Based  on  the  HEAO-1 
cost  data  available  and  the  above 
ground-rules,  a  program  cost  increase 
of  approximately  30%  could  be  expected 
for  a  single  observatory  program  if  the 
conventional  approach  is  used  for 
development , 


CONCLUSIONS 

While  the  procoflight  approach  is  recom¬ 
mended  for  scientific  type  space  probes, 
some  restraint  should  be  used  in  its 
application.  The  following  are  lessons 
learned  based  on  the  HEAO  experience 
to  date : 

1.  Define  the  program  and  base¬ 
line  the  requirements  for  the 
initial  invitation  for  bids. 
Late  changes  that  require 
reverification  can  substan¬ 
tially  increase  the  cost  of 
testing. 

2.  Give  special  attention  to 
newly  developed  items 
especially  where  the  state- 
of-the-art  is  concerned. 

The  new  items  generally 
require  a  qualification 
model  which  can  be  refurbished 
after  testing  and  used  as  a 
spare.  Extreme  caution  must 
be  exercised  in  using  off-the- 
shelf  designs  and  hardware 
especially  when  considering 
application  and  reliability. 

3.  Give  special  attention  to 
early  scheduling  and  prompt 
delivery  by  the  contractor 
of  test  specifications, 
procedures,  and  integration 
criteria  which  require 
coordination  and  review. 

4.  Maintain  good  communications 
among  all  participating  parties 
particularly  in  the  interface 
areas.  This  may  help  reduce 
the  overall  testing  by  making 
each  test  meaningful. 


5. 


Use  selected  screened 
electronic  parts  to  avoid 
unwarranted  early  failures. 
This  prevents  much  retesting 
resulting  from  rework. 

6.  Use  only  with  contractors 
that  are  well  experienced  In 
space  activities  and  that 
have  adequate  technical 
resources  to  assure  continuity 
to  the  end  of  the  program. 
Contractors  who  have  exper¬ 
ienced  failures  and  successes 
and  have  analyzed  the  dif¬ 
ferences  are  likely  to  have 
the  unique  knowledge  to 
routinely  develop  hardware 
which  will  operate  success¬ 
fully  the  first  time. 

7 .  Maintain  an  up-to-date 
materials  and  parts  list  of 
space  proven  Items.  Select 
as  much  as  possible  from  that 
list  In  order  to  avoid  new 
developments . 

8.  Use  a  conservative  design 
approach  to  allow  adequate 
margins  for  test  and  flight 
environments . 

Based  on  experience  gained  during  ground 
test  and  mission  operations  on  two  sets 
of  HEAO  hardware.  It  Is  concluded  that 
the  protofllght  approach  can  be  used 
successfully  with  attending  cost  savings 
and  without  any  compromise  In  reliability. 
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HEAD  EXPERIMENT  VERIFICATION  REQUIREMENTS 


TABLE  I,  HEAO  EXPERIMENT  VERIFICATION  REQUIREMENTS  (ConcludedJ 


TABLE  II,  HEAO  OBSERVATORY  Q/A  VERIFICATION  REQUIREMENTS 


TABLE  U,  HEAO  OBSERVATORY  Q/A  VERIFICATION  REQUIREMENTS  (Concluded) 
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r’lGURE  1,  Hfc;AO  HARDWARE  b  VERIFICATION  FLOW  DIAGRAM 
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FIGURE  2.  VERIFICATION  TEST  FLOW 
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FIGURE  3  SAMPLE  MANUFACTURING  &TEST  SCHEDULE 


VOYAGER  IMAGING  SYSTEM 

Fred  E.  Vescelus 
Robert  F.  Lockhart 
Jet  Propulsion  Laboratory 
4800  Oak  Grove  Drive 
Pasadena,  California  91103 

Abstract 

The  Voyager  spacecraft  imaging  instru¬ 
ment  has  been  a  system  development  by 
JPL,  conducted  from  1972  through  1976  to 
support  scientific  explorations  at  Ju¬ 
piter,  Saturn  and  Uranus.  Its  images  of 
the  Jovian  satellites  and  Jupiter  itself 
have  been  shown  in  three-color  illustra¬ 
tions  throughout  the  world  during  1979. 

Mission  reliance  on  this  instrument  to 
function  reliably  on  two  spacecraft  at 
hundreds  of  millions  of  miles  from  Earth 
and  years  after  launch  placed  stringent 
demands  on  the  reliability  aspects  of  the 
instrument  development.  Detailed  tech¬ 
nical  knowledge  and  understanding  of  every 
aspect  of  imaging  instrument  function  and 
performance  were  sought  to  defend  against 
risks  in  the  Voyager  missions. 

Introduction 

In  1973  NASA  and  Congress  authorized 
the  Voyager  mission  to  Jupiter  and  Saturn. 
This  mission  was  planned  with  two  space¬ 
craft,  each  carrying  both  a  long  and  a 
short  focal  length  television  system. 

These  systems  were  required  to  perform 
reliably  over  a  mission  lifetime  of  ten 
years,  with  one  of  the  spacecraft  targeted 
to  the  planet  Uranus,  for  a  total  mission 
flight  time  of  two  years. 

Experiment  Description 

The  Voyager  imaging  science  subsystem 
consists  of  two  cameras,  a  wide  angle 
camera  and  a  narrow  angle  camera.  Each 
camera  consists  of  an  optics  subassembly 
that  includes  an  eight-position  filter 
wheel  with  position  identification  sen¬ 
sors,  a  focal  plane  shutter  assembly  tiiat 
Includes  light  flood  lamps,  a  camera  head 
that  includes  a  magnetically  focused  and 
deflected  slow  scan  vidicon  for  the  con¬ 
version  of  optical  images  to  electronic¬ 
ally  processible  data,  a  support  electron¬ 
ics  subassembly,  and  a  power  supply  sub- 
assembly. 


The  wide  angle  camera  has  a  200  mm  fo¬ 
cal  length  lens  operating  at  a  fixed  rel¬ 
ative  aperture  of  f/3.5  and  a  field  of 
view  of  56  x  56  .^rad .  The  narrow  angle 
camera  contains  a  catadioptric  telescope 
with  a  1500  mm  focal  length  lens  opera¬ 
ting  at  a  fixed  relative  aperture  of 
f/8.5  and  a  field  of  view  of  7.4  x  7.4 
urad.  The  timing  and  control  functions 
are  contained  in  the  spacecraft  flight 
computer  and  are  brought  to  the  cameras 
by  a  set  of  18  interface  lines  for  each 
camera.  In  addition,  a  set  of  5  inter¬ 
face  circuits  present  the  digital  video, 
filter  position,  analog  engineering  tel¬ 
emetry  and  temperature  sensors  on  board 
to  the  computer  for  processing  and  for¬ 
matting  . 

The  video  data  consists  of  800  pixels 
per  line  and  800  lines  per  frame.  Each 
pixel  is  encoded  to  8  bits  to  provide 
256  shades  of  gray.  Therefore,  there 
are  5,120,000  bits  per  picture  which  are 
converted  at  a  rate  of  172.8  kbps  and 
are  transmitted  at  a  rate  of  115.2  kbps. 
It  requires  48  seconds  to  read  out  a 
picture  of  the  picture  transmitted  rate. 
Each  camera  has  several  modes  of  opera¬ 
tion:  normal  exposure  mode,  which 
shutters  the  cameras  at  one  of  the  avail¬ 
able  exposure  times  from  5  ',is  to  15.3  s; 
a  long  exposure  mode  which  provides  ex¬ 
posures  in  multiples  of  48  seconds;  a 
simultaneous  exposure  mode  whereby  each 
camera  is  shuttered  at  the  same  time; 
and  an  automatic  exposure  mode  whereby 
the  exposure  is  determined  by  the  content 
of  the  previous  exposure,  corrected  for 
the  filter  that  was  used  and  the  desired 
filter  for  the  picture  which  is  to  be 
taken. 

In  addition  to  these  exposure  modes, 
there  are  5  read  out  modes:  1:1,  2:1, 
3:1,  5:1,  and  10:1.  In  these  modes  the 
picture  is  stored  on  the  vidicon  and  read 
out  at  the  lower  data  rates  indicated  bv 
the  above  ratios.  There  are  also  two 
basic  edit  modes  whereby  a  partial  pic¬ 
ture  can  be  read  out  at  full  resolution 
containing  either  3/4  or  1/3  of  full 
frame,  or  a  full  frame  containing  either 
1/5  or  1/10  of  the  total  pixels.  Both  of 
the  above  slow  scan  and  the  edit  modes 
are  incorporated  to  accommodate  reduced 
telemetry  capabilities  while  providing 
real  time  pictures. 


477 


System  Design  Constraints 

The  design  constraints  for  the  Voyager 
television  system  may  be  divided  into 
three  basic  areas:  1)  science  require¬ 
ment  constraints,  2)  mission  constraints, 
and  3)  spacecraft  constraints. 

The  science  requirements  constraints 
dictate  primarily  the  performance  param¬ 
eters  such  as  focal  length,  focal  ratio, 
MTF  and  spectral  range,  nqise  performance, 
digital  resolution,  optical  filters, 
shutter  speed,  etc. 

The  mission  constraints  impact  the 
design  in  the  area  of  lifetime  require¬ 
ments  and  environmental  requirements. 

For  the  Voyager  mission  these  were  the 
roughest  requirements  to  meet.  The 
operational  lifetime  requirement  was  for 
ten  years  unattended.  The  radiation 
environment  at  Jupiter  placed  the  most 
severe  requirements  on  the  selection  of 
parts  and  material.  It  was  required  that 
the  imaging  system  operate  after  receiv¬ 
ing  a  total  dose  of  60  K  Rads  and  oper¬ 
ate  with  little  or  no  Interference  while 
receiving  a  fluence  of  2.5  x  lo''^  elec¬ 
trons  per  cm2,  spacecraft  constraints 

imposed  weight  limits,  power  limits, 
vibration  levels,  magnetic  levels,  EMI 
and  EMC  levels,  and  an  environmentally 
related  requirement  for  an  equal-potential 
spacecraf  t . 

The  Design  Process 

At  the  onset  of  the  program  all  of  the 
known  constraints  (though  sometimes  not 
yet  quantified)  were  considered  and  a 
system  was  defined  around  as  many  known 
and  proven  elements  as  possible.  For  ex¬ 
ample,  the  N.  A.  optics  was  the  same 
design  of  the  optics  used  on  Mariner 
Venus  Mercury.  The  shutter  and  filter 
wheel  mechanisms  were  third  or  fourth 
generation  mechanisms  which  had  a  proven 
lifetime  and  reliability.  JPL  was  in  the 
throes  of  an  extensive  electronic  parts 
characterization  for  radiation  damage.  A 
few  selected  I.C. 's  were  also  undergoing 
I  a  radiation  hardening  development.  This 

provided  a  very  restrictive  list  of 
electronic  components  from  which  to  design 
the  required  circuits.  Having  gone 
through  this  process,  components  or  ele¬ 
ments  were  identified  which  had  known 
I  problems  or  some  unknown  aspect,  and  ex¬ 

tensive  test  and  qualification  programs 
were  initiated  where  adequate  substitutes 


could  not  be  found.  For  example,  the 
vidicon  Itself  proved  to  be  the  weakest 
link  for  lifetime  in  the  system.  A  pro¬ 
gram  was  Initiated  to  develop  a  new  fil¬ 
ament  and  cathode  which  would  provide  for 
the  required  lifetime.  Further,  a  cir¬ 
cuit  was  added  to  the  system  to  allow 
small  changes  to  the  grid  voltage  of  the 
vidicon  which,  due  to  the  electron  optics 
of  the  tube,  would  allow  a  new  spot  on 
the  cathode  to  be  used  when  the  cathode 
current  degraded.  Tests  also  had  to  be 
devised  to  characterize  the  radiation 
Interference  performance  of  the  vidicon, 
the  signal  chain,  and  the  optical  ele¬ 
ments  of  the  system. 

The  next  step  was  to  compile  a  compre¬ 
hensive  set  of  derating  guidelines  for 
all  components  to  be  used  in  the  system. 
This  included  stress  limits,  part  tol¬ 
erance,  aging  effects,  tolerance  on  the 
rated  value,  thermal  derating  and  radia¬ 
tion  derating.  Radiation  derating  was 
continuously  updated  as  the  design 
process  progressed.  This  was  due  to  pri¬ 
marily  two  factors:  1)  many  components 
were  not  yet  characterized  at  the  time 
the  design  was  initiated,  and  2)  Pioneer 
flew  by  Jupiter  which  impacted  the  radia¬ 
tion  model  being  used.  The  next  step  in 
the  process  was  to  set  design  priorities 
for  tradeoff  criteria.  The  set  selected 
were:  design  problems  would  be  solved 
first  by  circuit  design  on  component 
selection,  or  in  the  case  of  mechanical 
or  optical  design  by  material  selection; 
second,  in  the  case  of  radiation  or  ther¬ 
mal  problems  by  location  within  the  sub¬ 
system;  and  third,  by  external  shielding 
or  spot  shielding  for  radiation  effects. 

It  was  the  careful  attention  to  these 
concepts  and  design  criteria  that  defined 
the  final  system.  For  example,  the 
timing  and  control  function  resided  in 
the  spacecraft  Flight  Data  System  (FDS) 
primarily  because  the  FDS  was  a  redundant 
system,  whereas  the  television  was  not. 
The  W.  A.  optics,  which  was  a  totally  new 
design,  was  designed  around  radiation 
hard  glass.  One  other  very  stringent  rule 
that  was  applied  was  that  no  design  change 
was  allowed  to  take  place  without  first 
investigating  the  entire  system  for  hid¬ 
den  impacts. 

Keys  to  Reliability  and  Performance 
Assurance 

Many  specific  items  were  combined  to 


provide  reliability  and  performance 
assurance.  For  discussion  purposes,  three 
phases  of  hardware  activity  will  be  con¬ 
sidered:  the  conceptual  design  phase, 
the  detailed  design  phase,  and  the  design 
verification  and  qualification  phase. 

The  key  element  of  the  conceptual  de¬ 
sign  phase  is  the  configuration  of  the 
Instrument.  In  determining  the  config¬ 
uration,  the  following  issues  were 
addressed : 

.Instrument  functional  split  with 
spacecraft 

.Spacecraft  Interfaces 

.Instrument  Independency 

.Instrument  redundancy 

Each  of  these  items  were  examined  early 
in  the  program  to  Insure  a  rational,  re¬ 
liable,  and  testable  Instrument  config¬ 
uration.  Often  times,  these  issues  came 
head  to  head  with  existing  spacecraft 
constraints  and/or  monetary  constraints. 
Compromises  were  then  made  on  both  sides. 
For  example,  full  instrument  redundancy 
would  provide  the  greatest  assurance  for 
success,  but  was  ruled  out  on  individual 
spacecraft  after  consideration  of  the  dual 
spacecraft  launch.  Instead,  the  wide 
angle  and  narrow  angle  cameras  were  made 
independent  of  each  other.  The  function¬ 
al  split  between  the  television  system 
and  the  spacecraft  flight  data  system 
(FDS)  was  also  made  at  this  phase  of  the 
design,  with  the  logic  control  functions 
coming  from  the  FDS  and  the  digital  data 
flowing  to  the  FDS. 

The  next  design  phase  was  the  detailed 
design.  Considered  during  this  activity 
were: 

.Circuit  design  constraints 

.Mechanical  design  constraints 

.Optical  design  constraints 

.Proven  approaches 

.Qualified  parts  selection  lists 

.Failure  modes  and  effects 
criticality  analysis 

.Worst  case  analysis 


.Configuration  control 

.Design  reviews 

These  items  were  all  key  elements  and 
were  all  used  to  both  control  and  in¬ 
fluence  the  design  activity.  A  detailed 
design  constraints  document  was  generated 
at  the  beginning  of  the  detailed  design 
phase  to  control  and  influence  the  over¬ 
all  design.  Past  approaches  proven 
effective  were  Incorporated,  wherever 
possible  and  past  concepts  relied  upon 
for  the  new  design  work.  Space  and  ra¬ 
diation  qualified  parts  were  compiled 
into  lists  and  parts  selected  for  design 
purposes  came,  where  possible,  from  these 
lists.  Where  parts  could  not  be  selected 
from  an  approved  list,  qualification  of 
the  selected  parts  took  place.  A  failure 
modes  and  effects  analysis  was  performed 
on  all  circuit  and  instrument  intakes, 
and  the  design  was  changed  where  appro¬ 
priate.  This  analysis  was  particularly 
important  as  Instrument  interfaces. 

Worst  case  analysis  was  performed  on  all 
circuit  designs  to  a  Voyager  project 
approved  set  of  component  tolerance  var¬ 
iations.  This  set  of  analyses  was  es¬ 
pecially  important  for  the  Voyager  tel¬ 
evision  system  as  it  provided  the  only 
inclusive  way  to  determine  performance 
of  the  instrument  under  radiation.  Con¬ 
figuration  control  and  design  reviews 
were  used  throughout  the  design  phase  to 
both  keep  track  of  the  design  and  control 
its  evolution  and  to  insure  that,  in  the 
case  of  the  design  reviews,  to  provide  an 
overview  activity,  provide  additional,  im¬ 
partial  expertise,  and  other  constructive 
guidance.  All  of  these  items  were  impor¬ 
tant  in  the  reliable  and  understood  design 
and  expected  operation  of  the  Imaging  in¬ 
strument  . 

Following  the  detailed  design  activity, 
the  design  verification  and  qualification 
phase  was  undertaken.  This  phase  was  an 
extremely  Important  one,  as  it  empirically 
verifies  or  refutes  the  design  assumptions 
made  during  the  detailed  design  phase. 
Included  in  this  effort  were  parts  qual¬ 
ifications,  subsystem  testing,  instrument 
qualification,  and  system  testing. 

Some  of  the  items  considered  during 
this  phase  were: 

.Parts  qualification 

•Vidlcon  qualification 


479 


•Circuit  testing 
•Subsystem  testing 
•Instrument  qualification 
•Design  reviews 
•Problem  and  failure  reports 
•Configuration  management 
•System  testing 

All  of  these  items  were  of  importance  and 
contributed  greatly  to  the  success  of  the 
overall  effort.  In  support  of  this,  a 
hardware  compliment  that  consisted  of  de¬ 
velopmental  assemblies,  a  breadboard, 
prototype  for  design  qualification,  flight 
spare,  and  flight  units  was  utilized. 

Parts  qualification  was  vital;  many  of 
the  components  used  did  not  initially 
have  radiation  performance  history.  This 
information  was  vital  to  confirm  the  de¬ 
sign  margins  used  in  the  design  and  anal¬ 
ysis.  The  testing,  both  circuit  and  sub¬ 
system,  provided  both  performance  data 
and  verification  of  key  parts  of  the 
analysis.  In  addition,  many  key  things 
such  as  noise  performance,  not  readily 
amenable  to  exact  analysis,  were  checked. 
Testing  at  this  level  was  also  performed 
over  temperature  and  voltage  extremes, 
and  selected  subassemblies  experienced 
radiation,  vibration,  and  thermal  vacuum 
testing. 

The  subassemblies  then  were  combined 
and  went  into  a  flight  environmental  mar¬ 
gin  and  performance  test  sequence  to 
verify  the  complete  instrument  performance. 

From  this  point,  integration  and  testing 
on  the  spacecraft  was  used  to  validate 
the  spacecraft/television  Interface  and 
the  Interface  design  assumptions  used. 
Supporting  these  efforts  were  design  re¬ 
views  at  initial  points  (preliminary,  de¬ 
tailed,  and  pre-shipment  formal  reviews 
were  held,  along  with  a  series  of  inform¬ 
al  design  reviews),  configuration  manage¬ 
ment  of  the  design  requirements  and 
drawings,  a  problem  failure  and  reporting 
system,  and  weekly  project  meetings. 

Approach  Considerations 

The  design  approach  used  on  the  Voyager 
mission  for  the  television  system  is 
essentially  the  same  as  that  used  on 


previous  JPL  spacecraft  in  the  Mariner 
and  Surveyor  and  Viking  series.  This 
approach  has  been  found  to  be  very  effec¬ 
tive  in  terms  of  identifying  design  prob¬ 
lems  and  in  terms  of  verifying  analytic¬ 
ally  design  margins  for  the  hardware  use. 
In  the  case  of  the  Voyager  activity,  four 
years  were  expended  in  going  through  first 
the  subsystem  requirements  generation, 
the  detailed  design  of  the  instrument, 
the  analysis  to  support  the  design,  and 
then  the  testing  to  verify  that  the  anal¬ 
ysis  and  the  design  practices  themselves 
were  sound.  This  approach  has  been  very 
productive  and  all  phases  of  the  approach 
outlined  have  identified  and  eliminated 
potential  failure  and  performance  limita¬ 
tion  mechanisms  in  the  design.  A  dis¬ 
advantage  to  this  approach  is  the  length 
of  time  required  to  go  through  these 
processes  and  the  expense  Involved, 
particularly  in  doing  a  sophisticated 
worst  case  analysis  and  a  detailed  test 
program  with  the  hardware.  In  the  case 
of  the  Voyager  mission,  these  were  vital 
keys  to  the  overall  reliability  obtained 
with  an  instrument  that  was  not  redundant 
on  the  spacecraft  launch  of  long  duration. 
Without  the  confidence  generated  by  these 
approaches,  certainly  the  requirements  of 
a  long  lifetime  deep  space  mission  could 
not  be  adequately  met. 

Conclusions 

This  paper  describes  in  some  detail  the 
design  process  undertaken  for  the  Voyager 
imaging  system  on  the  current  Voyager 
spacecraft  mission  to  the  planets  Jupiter, 
Saturn  and  Uranus.  This  approach  follows 
a  systematic  design,  development  and 
testing  activity  aimed  at  eliminating  de¬ 
sign  and  manufacturing  defects  and  ver¬ 
ifying,  to  the  extent  possible,  perform¬ 
ance  anticipated  at  the  spacecraft  en¬ 
counters.  Although  fairly  expensive  in 
terms  of  time,  effort,  and  money,  this 
approach  has  proven  effective  in  assuring 
reliability  under  long  duration,  environ¬ 
mentally  difficult  conditions. 
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SCieNCE  OBJECTIVES  OF  MJS 
//•.•iCrC^lE^CE  If.^ESTIGATiO’, 


Key  Elements 


OBTAIN  global  COLOR  PHOTOGRAPHY  OF  JUPITER  AND  SATURN.  AT  BEHER  THAN 
EARTH-BASED  RESOLUTION,  OVER  AN  EXTENDED  PERIOD. 

MEASURE  THE  MOTIONS  0*^  SMALL  CLOUD  FEATURES  TO  RELATE  LOCAl  WIND 
CHARACTERISTICS  TO  GLOBAL  DYNAMICS. 

OBSERVE  vertical  STRATIFiCATlON  IN  THE  ATMOSPHERES  OF  JUPITER,  SAir- 
TIT.-fJ  AND  OTHER  BODIES  WITH  SUBSTANTIAL  ATMOSPHERES , 

ACCURATELY  CHARACTERIZE  THE  OPTICAL  PROPERTIES  OF  THE  PLANETS  AND  'VJOP 
SATELLITES,  OVER  A  WIDE  SPECTRAL  RANGE.  AFTER  A  FOUR-YEAR  FLIGHT. 

INVESTIGATE  COLOR  VARIATIONS  IN  THE  PLANETARY  ATMOSPHERES  AND  ON  THE 
satellite  SURFACES. 

OBTAIN  HIGH  RESOLUTION  I  !  kml.  LOW  SATEAR  IMAGES  OF  SEVERAL  SATELLITES 
TO  CHARACTERIZE  SURFACE  GEOLOGIC  STRUCTURE. 

MEASURE  THE  FIGURE  AND  SPIN  AXIS  ORIENTATION  OF  THE  MAJOR  SATELLITES. 

DETERMINE  THE  SPATIAL  DISTRIBUTION  OF  AAETHANE  AND  ATOMIC  SODIUM  ON  THE 
PLANETS  AND  10.  RESPECTIVELY. 

OBTAIN  HIGH  RESOLUTION  I  I  kmi.  LOW  SMEAR  IMAGES  OF  THE  RINGS  OF  SAT'IRIL 

OBSERVE  THE  DARK  SIDE  OF  JUPITER  TO  SEARCH  FOR  LIGHTNING.  AURORAE.  O’’ 
METEORS. 

MINIMIZE  COVERAGE  LOSSES  DUE  TO  POINTING  ERRORS. 

MAXIMIZE  THE  EFFICIENCY  OF  IMAGING  DATA  RETURN  AT  SEVERAL  DATA  RATES. 


•Circuit  Design  Constraints 

•  Mechanical  Design  Cctnstraints 

•  Optical  Design  Constraints 

•  Proven  Approaches 
•Qualified  Parts  Selection  Lists 

•  Failure  Modes  and  Effects 
Criticality  Analysis 

•  Worst  Case  Analysis 
•Configuration  Control 

•  Design  Reviews 


Key  Elements 


•Instrument  Functional  Split 
•Spacecraft  Interfaces 
•Instrument  Independency 


•  Parts  Qualification 

•  ViDicoN  Qualification 
•Circuit  Testing 
•Subsystem  Testing 
•Instrument  Qualification 

•  Design  Reviews 

•Problem  and  Failure  Reports 
•Configuration  Management 

•  System  Testing 
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V0YA6£R  (HA6ING  SYSTEfl 


AMD  SHUTTER  MECHAMISHS 
TEST  PftOGKAM 


!•  Circuits  to  le  margin  tested  over  worst  case  combinations  oe: 
a)  DC  Voltage: 


HOLESXME  EMGINEERIMG  MODELS  ONLY  (F.W.  &  SHinTERS) 

FASTEX  MOVIES  —  ENGIMEERINC  MODUS  ONLY  (F.W.  &  SHirTTERS) 


EMVXROKKENrAL  (F.tf.  &  SHVrTERS) 

VUIAIICM  TO  T.A.  LEVELS  ENGIMEEE  (MXTS 
TKESMAL  TO  F.A.  LEVELS  —  ALL  IMITS 

UFS  TEST  -  COKTlNUOtB  CYCLES  (F.V,  S  SHUTTEIS) 
EN6D1EEUMC  MODELS  —  SO,  000  CYCLES 


Power  surrlt  tested  with  inrut  rower: 


2400  -101.  lOOv  P-P  i  IQY 


B>  TemreratuRE: 


Bus  Electronics  -50*C  to  ♦SS^C 
Camera  Assemblies  -SO'C  to  ♦AS^C 


Camera  Circuits  -?0*C  to  *70*C 


'  ‘sOURCE/lOAD  IMREOANCf  tEvELS 
Di  lN»tjT  Waveform  To(,erances 


ASSEMBLY  SCBEEMINC  (F.U.  4  SHUrTEBS) 

FBOrOTYPE  MODELS  —  20,000  CYCLES 
FLlCffT  IMITS  —  1,000  CYCLES 

OPERATING  CHARACmiSnCS  AND  CALIBRATION  (ALL  SHUTTERS) 
MEASURE  AMD  UODRD 
LICHI  LEAK 
A  BLADE  BOUia 

A  EXPOSURE  •  ACTUAL  TIME  vs .  COMtAMD  TIME 
▲  BLADE  SPEED  AMD  REPEATABIUTT 

(TRAVEL  TIME,  DELAY,  AMD  ACCELEBATIOHS) 

A  SHADZMC 
A  HABCXM  TESTS 

(KZNZMUH  VOLTAGE,  CAPAaTANCE,  AMD  PULSE  WIDTH) 


pAn**<€Tj,ns  and  Recorded 

»'  'RiriCRl.  OUTPUT  PARAMETERS 


R  t  ®OmE  »  0  I SS I RRT I  ON 


)  t^MASE  ’^ARr,  I  N 

0)  Internal  levels 


E 1  Internal  logic  levels 


E)  Survival  and  Recoverv  time  under  ohtrut  smort  circuit 
AND  INRiir  OVER  voltage  CONDITIONS  WHERE  ARPlICARlE* 


FILTER  WHEEL  (ALL  WITS) 
BACIXASR 

VERIFY  I.D.  POSITION  CODING 
MARGIN  TESTS 


4  TESTS  REPEATED  IN  INCREMENTS  THROUGH  F.A,  TDtPERATURE  RANGE. 


W»E-AN61E  1EN5  BARREL 
DESIGN  CONSTRAINTS 


V'OiCON  wOMPQNINT  I 

RARTS  ! 


I  Cl^  *SST  ;  MICH  ASSYl 


I  VAC.  FIRl  X-RAV 


ttAlINC  M  MB  INSRCCTKIN 


l-RAT  H  ^  r*  0^ 


DESIGN  ENVaOPE 
WEIGHT 

MINIMUM  BACK  FOCUS 
DEPTH  OF  FOCUS 
OPERATMG  TEMPERATURE  RANGE 
TA  VIBRATION  UVaS 


RAOiAnON  ENVIRONMENT 


MAGI^  MATERIALS 


}  RiCtRMUM  1 
;  RCBMMMAMa  0«CX  f 


UHn.  LONG  BY  BHn.  SQUARE 
2.S  lbs 
LI  In. 

±0.001  In. 

-ro®  TO  +40®C 

SINUSOIDAL  VIBRATION  LIVELS 

S  TO  12  Hz  1.02  cm  DOUBU  AMPUTUDE 

12  TO  30  Hz  2.0CrfflS 

30  TO  100  Hz  B.0 

100  70  2000  Hz  4.5 

RANDOM  VIBRATION  11. 1  G  rms  OVERAU 

1 1  10^  Blfctrons/cm^ 

3  MbV  Ntctrons 

LIMIT  USE  OF  MACNEHC  MATERIALS 


'  ACCtFTAMCf 
i  PWDMUHa  C>«CK  * 


ftIGHT  CANBlOAfl  I!  IW  MFfWOBlMAMal 
VIOICONS  n  TWRAIAl  MIA  i|  CNfCR  I 


SCRQNfO  FUCMT 
VIOICONS 


I  visual  WCMANICAi 
*  INSMCTION 
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WIDE-ANGLE  LENS  BARREL  PROBLEM  AREAS 


TA  TEST  SEQUENCE  (PROTOTYPE) 


RADIATION  ENVIRONMENT  SEVERE  RADIATION  ENVIRONMENT  LIMITS 
USE  Of  PREVIOUSLY  FLIGKT-QUALIflEO 
ORGANIC  MATERIALS 

MATERIALS  IN  QUESTION 

1.  DUROID 

2.  RULON  A 

3.  TEFLON 

4.  DRY  LUBRICANT  (ELECTROFILMI 

5.  CAT-A-LAC-FLAT  BLACK  PAINT 

6.  CHEMGLAZE  n06  FLAT  BLACK  PAINT  AND  PRIMER 

7.  VITON 

8.  OPTICAL  MATERIALS 


MJS  WIDE-ANGLE  LENS  DEVELOPMENTAL  TESTS 


•  ACCEPTANCE  TEST 

•  BENCH  CALIBRATION 

•  RADIATION 

•  SYSTEM  VERIFICATION 

•  MECHANICAL  TEST 

•  SYSTEM  VERIFICATION 

•  THERMAL  VACUUM  AND  CALIBRATION 

•  SYSTEM  VERIFICATION 

•  EMI /EMC 

•  MAGNETIC 


A.  OPTICAL  PERFORMANCE  TESTS 

1.  SYSTEM  WAVEFRONT 

2.  LIMITING  RESOLUTION 

3.  FOCAL  LENGTH 

4.  BACK  FOCAL  LENGTH 

5.  OPTICAL  AXIS  ALIGNMENT 

6.  T-STOP 

7.  VIELING  GLARE 

B.  VIBRATION  AND  ACOUSTIC  TESTS  (COMPLETE  WIDE-ANGLE  CAMERAI 

1.  LOW  LEVa  SINE  SWEEPS 

2.  FA  LEVa  SINE  AND  RANDOM  VIBRATION 

3.  TA  LEVEL  SINE  AND  RANDOM  VIBRATION 

4.  ACOUSTIC 

C.  THERMAL  -  VACUUM 

1.  THERMAL  FOCUS  SHIFT  MEASUREMENT 

2.  VACUUM /AIR  FOCUS  SHIFT  MEASUREMENT 

3.  TA  THERMAL  CYCLE  TEST 


FA  TEST  SEQUENCE  AND  CALIBRATION 
(FLIGHTS  1.  2.  AND  3) 

COMPONENT  LEVEL  PRIOR  TO  INTEGRATION  OF  OPTICS 
OPTICS 
FILTERS 
SHUHERS 
V  ID  ICONS 

•  ACCEPTANCE  TESTS 

•  BENCH  CALIBRATION  »1 

•  MECHANICAL  TEST 

•  SYSTEM  VERIFICATION 

•  THERMAL  VAC  +  2  CAL  I  BRAT  ION 

•  SUBSYSTEM  VERIFICATION 

•  BENCH  CALIBRATION  #2 

•  EMC /EMI  IF  REQUIRED 

•  MAGNETIC  IF  REQUIRED 

•  SYSTEM  VER  IF  ICAT  ION  (SAF) 

•  BENCH  CALIBRATION  #3  (PR  lOR  TO  DELIVERY  TO  ETR) 

•  SYSTEM  VERIFICATION  (ETR) 
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COST  EFFECTIVE  PARTS  SELECTION  CRITERIA 
Leland  D.  Swanson 
RELIABILITY  ENGINEER,  TRW/DSSG 

For  most  hardware  system  design  and 
development  programs,  the  requirements 
for  the  selection  of  parts  is  establish¬ 
ed  during  the  proposal  p^ase  when  very 
little  information  is  available.  For  a 
cost  effective  mission  assurance  program, 
it  is  essential  that  the  selection  of 
parts  be  compatible  with  the  system  rel¬ 
iability  requirements,  however,  the 
total  cost  of  the  mission  must  also  be 
minimized. 

In  this  presentation,  I  will  discuss  a 
parts  cost  trade-off  study  made  to  verify 
the  part  selection  criteria  for  the  MX 
Missile  Instrumentation  and  Flight  Safety 
System  (IFSS).  The  IFSS  is  not  Important 
to  my  purpose  in  this  presentation.  It 
merely  provides  the  vehicle  to  illustrate 
some  techniques  which  can  be  used  early 
in  a  design  and  development  program  to 
guide  management  in  making  cost  effective 
decisions  relative  to  part  selection 
criteria  and  mission  assurance. 

The  Instrumentation  and  Flight  Safety 
System  for  the  MX  missile  monitors 
missile  performance  during  flight  test. 

It  consists  of  airborne  instrumentation 
(ABI)  which  senses,  conditions,  and 
transmits  data  on  missile  performance, 
and  ground  based  support  equipment  (SE) 
to  receive  and  process  the  test  flight 
data.  The  ABI  also  contains  a  coramand- 
destruct  Flight  Safety  Subsystem  to  de¬ 
stroy  the  missile  should  it  become  nec¬ 
essary.  This  presentation  is  concerned 
only  with  the  ABI  portion  of  the  IFSS . 

Physically,  IFSS/ABI  components  are 
located  in  all  stages  of  the  missile  with 
the  main  elements  located  in  Stage  IV 
and  the  Reentry  System.  IFSS/ABI  com¬ 
ponents  include  items  such  as  multi¬ 
plexers,  power  supplies,  signal  con¬ 
ditioners,  a  receiver/decoder ,  trans¬ 
ponders,  batteries,  various  signal  pro¬ 
cessors  and  sensors,  amplifiers,  trans¬ 
mitters,  antennas  and  of  course  the  in¬ 
terconnecting  cabling. 

During  this  presentation,  we  will 


very  briefly  review  the  results  of  a 
parts  cost  versus  reliability  trade  study 
for  the  IFSS/ABI  and  cover  the  topics 
shown  on  Slide  1.  The  purpose  of  the 
study  was  to  resolve  a  recurring  manage¬ 
ment  question  as  to  the  appropriateness 
of  the  part  selection  criteria  Imposed 
on  the  IFSS/ABI. 

The  IFSS/ABI  reliability  requirements  are 
summarized  on  Slide  2.  The  Telemetry 
G&C  Data  Link  and  the  Plight  Safety  Sub¬ 
system  (FSS)  flight  termination  function 
are  considered  critical  for  safety  rea¬ 
sons,  therefore,  the  prelaunch  and  flight 
reliability  requirements  for  these  funct¬ 
ions  are  specified  separately. 

For  the  purpose  of  our  study,  we  divided 
the  alternatives  for  part  selection  into 
four  categories  as  shown  on  Slide  3.  The 
part  selection  criteria  is  based  on 
SAMSO-STD-77-7  which  is  the  control  doc¬ 
ument  for  standardizing  parts,  mater¬ 
ials  and  processes  for  the  MX  missile. 

The  AVE/OSE  category  of  parts  consists 
of  parts  procured  to  the  best  available 
specifications  which  include  special  pro¬ 
visions  for  process  baseline  controls, 
parameter  drift  screens,  and  destructive 
physical  analyses.  The  ABI  category  of 
parts  is  composed  of  high  and  medium 
grade  military  spec  parts.  The  TSE/MSE 
category  is  composed  of  lower  grade  mil- 
spec  parts  while  the  category  on  the 
right  consists  of  parts  procured  to 
nothing  more  than  suppliers  part  numbers 
or  commercial  catalog  information. 

The  assumptions  and  conditions  used  to 
establish  the  relationships  between  parts 
cost  and  failure  rates  are  shown  in  Slide 
4.  To  establish  the  parts  cost, 
prices  were  obtained  from  a  number  of 
sources  using  a  base  quantity  of  1,000. 
Failure  rate  Information  was  obtained 
from  actual  experience  on  the  MINUTEMAN 
program  and  from  MIL-HDBK-217 . 

The  results  of  the  parts  cost  vs.  failure 
rates  portion  of  the  study  is  shown  on 
Slide  5.  It  is  Interesting  to  note 
that  for  electronic  parts,  two  orders  of 
magnitude  improvement  in  reliability  can 
be  obtained  for  a  relatively  small  in¬ 
crease  in  parts  cost.  The  relationships 
between  parts  cost  and  failure  rates 
shown  In  Figure  I  forms  the  bests  for 


expanding  the  cost  trade-off  study  to 
the  IFSS/ABI  component  and  system  level. 

The  assumptions  and  conditions  used  to 
expand  the  parts  cost  vs.  failure  rate 
study  to  the  IFSS  Master  Multiplexer 
Unit  (MUX)  are  shown  In  Slide  6.  The 
MUX  electronic  parts  cost  and  probability 
of  failure  are  derived  from  the  parts 
cost  and  failure  rate  information  of 
Figiire  I,  multiplied  by  the  MUX  parts 
count. 

The  results  of  the  parts  cost  vs.  rel¬ 
iability  trade  study  on  the  MUX  is  shown 
In  Slide  7.  The  MUX  risk  cost  was  cal¬ 
culated  by  multiplying  the  estimated  cost 
of  the  MUX  by  Its  probability  of  failure. 
The  composite  cost  curve  is  simply  the 
sum  of  the  electronic  parts  cost  and  the 
MUX  risk  cost.  Figure  II  Indicates  that 
if  one  considered  the  MUX  alone  for  pre¬ 
launch  operation,  the  optimum  choice  of 
parts  from  a  cost  standpoint  would  be  the 
TSE/MSE  category  of  parts,  l.e.,  lower 
grade  military  spec  parts.  It  also  ill¬ 
ustrates  that  the  use  of  commercial  parts 
would  not  be  cost  effective. 

However,  to  determine  the  optimum  sel¬ 
ection  of  parts  for  the  MUX  in  prelaunch 
operation  is  not  our  objective.  Our 
objective  is  to  determine  the  optimum 
parts  selection  criteria  for  the  entire 
IFSS/ABI  system  considering  both  pre¬ 
launch  operation  and  the  flight  phase  of 
the  missile  flight  test.  To  accomplish 
this  objective,  the  additional  assumpt¬ 
ions  shown  in  Slide  8  were  made.  The 
mission  recycle  cost  is  defined  as  the 
cost  of  interrupting  a  flight  test  be¬ 
cause  of  an  IFSS/ABI  failure,  finding 
and  repairing  the  cause,  and  recycling 
the  flight  test  sequence.  In  the  worst 
case,  this  would  include  cancelling  and 
rescheduling  the  test  flight. 

The  results  of  expanding  the  parts  cost 
vs.  reliability  trade  study  to  the  IFSS/ 
ABI  System  level  considering  prelaunch 
operations  only,  is  shown  in  Slide  9. 
These  results  were  derived  in  a  manner 
similar  to  that  used  for  the  MUX  des¬ 
cribed  previously.  Three  different 
curves  are  shown  which  represent  the  risk 
cost  associated  with  the  lower,  medium 
and  higher  estimated  mission  recycle 
cost. 


Slide  10  shows  the  same  information  pre¬ 
sented  as  composite  costs,  l.e.,  the 
parts  cost  and  the  mission  recycle  risk 
costs  are  summed.  The  composite  cost 
curve  permits  one  to  establish  the  min¬ 
imum  cost  point  and  to  determine  the 
optimum  parts  selection  category.  Using 
the  lowest  estimate  for  mission  recycle 
cost,  the  location  of  the  minimum  cost 
point  indicates  that  the  selection  of 
TSE/MSE  parts  would  be  appropriate  for 
prelaunch  checkout  operation  of  the  IFSS/ 
ABI.'  For  the  medium  and  higher  estimate 
of  mission  recycle  costs,  the  selection 
of  ABI  category  of  parts  would  be  more 
appropriate. 

The  results  of  the  parts  cost  vs.  rel¬ 
iability  study  on  IFSS/ABI  for  the  flight 
phase  of  the  mission  is  shown  in  Slide  IL 
The  assumption  has  been  made  that  an 
IFSS/ABI  failure  could  result  in  a  loss 
of  the  mission  ranging  from  5%  to  100%, 
i.e.,  a  IFSS/ABI  part  failure  could  re¬ 
sult  in  only  a  small  loss  of  data  or  a 
complete  loss  of  the  mission  depending 
upon  where  the  failure  occurred  in  the 
system  and  the  time  during  the  flight  at 
which  it  occurred. 

Slide  12  shows  the  now  familiar  composite 
curve  of  parts  cost  and  mission  risk  cost 
as  a  function  of  IFSS/ABI  probability  of 
failure  for  the  flight  portion  of  the 
mission  considering  a  5%,  20%  and  50% 
loss  of  the  mission  data.  It  also  shows 
the  minimum  cost  and  optimum  selection  of 
part  quality  assurance  levels  considering 
only  the  flight  portion  of  the  mission. 

To  complete  the  parts  cost  vs . probability 
of  failure  study  for  the  entire  mission, 
the  study  results  from  the  prelaunch  and 
flight  portions  of  the  mission  must  be 
combined.  This  combination  is  illust¬ 
rated  in  the  next  three  slides  as  Figures 
VII,  VIII  and  IX. 

Figure  VII  shows  the  results  of  the  study 
considering  the  entire  mission  using  the 
most  conservative  assumptions,  i.e.,  the 
lowest  estimates  for  missile  recycle  cost 
and  loss  of  flight  data  as  a  conseq¬ 
uence  of  a  IFSS/ABI  part  failure. 

Under  these  assumptions.  Figure  VII  in¬ 
dicates  thac  the  optimum  selection  of 
parts  for  the  IFSS/ABI  would  be  the 
TSE/MSE  category  of  parts,  i.e.,  lower 
grade  mil-spec  parts. 
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Figure  VIII  shows  the  results  of  the 
study  for  the  entire  mission  using  a 
medium  estimate  for  missile  recycle  cost 
($700K)  and  a  20%  mission  loss  of  flight 
test  data.  Figure  VIII  Indicates  that 
using  a  mix  of  the  higher  and  lower  grade 
mil-spec  parts  would  be  the  most  cost 
effective  under  these  assumptions. 

Figure  IX  illustrates  the  result  of  the 
study  using  the  higher  estimate  of 
missile  recycle  cost  ($21tf)  and  a  50% 
mission  loss  of  test  flight  data.  The 
figure  indicates  that  the  optimum  part 
selection  criteria  under  these  assumpt¬ 
ions  would  be  ABI  parts,  l.e.,  higher 
grade  mil-spec  parts. 

The  question  now  arises  as  to  what  level 
of  parts  are  required  for  the  IFSS/ABI 
system  to  meet  the  reliability  require¬ 
ments  as  identified  on  Slide  2. 

The  matrix  on  Slide  16  shows  the  results 
of  reliability  predictions  based  on  the 
failure  rates  for  the  ABI  category  of 
parts  as  determined  from  M1L-HDBK-217C. 

The  predictions  Indicate  that  all  of  the 
IFSS/ABI  reliability  requirements  will 
be  met  using  parts  selected  from  Che  ABI 
category. 

The  matrix  on  Slide  16A  shows  the  results 
of  a  similar  prediction  using  the  fail¬ 
ure  rates  for  the  TSE/MSE  category  of 
parts.  The  prediction  clearly  indicates 
that  the  IFSS/ABI  system  will  not  meet 
the  reliability  requirements  with  parts 
selected  from  the  TSE/MSE  category  of 
parts. 

Conclusions  and  recommendations  from  the 
study  are  summarized  on  Slide  17  and  18 
respectively.  The  study  confirmed  that 
the  parts  selection  criteria  specified 
for  the  IFSS/ABI  was  optimum  from  a  cost 
standpoint  and  satisfies  the  system  rel¬ 
iability  requirements. 
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TESTING  APPROACH  FOR  MX  MISSILE 
INERTIAL  MEASUREMENT  UNIT 

William  A.  Lockie,  Manager 

IMU  Hardware  Project  Engineering  Branch 
Northrop  Electronics  Division 


The  purpose  of  this  presentation  Is  to 
show  how  the  test  methods  and  test  equip¬ 
ment  for  the  MX  Missile  Inertial  Measure¬ 
ment  Unit  (IMU)  are  being  determined. 
There  are  five  parts  to  the  presentation. 
The  first  two  parts  are  a  description  of 
the  IMU.  The  third  part  discusses  the 
methodology  being  used  to  develop  the 
test  methods  and  test  equipment.  The 
fourth  part  covers  the  status  of  test 
methods  and  test  equipment  at  the  comple¬ 
tion  of  the  Advance  development  program 
(ADP)  and  the  start  of  the  F^uH  d^ale 
Engineering  development  (FSED)  program. 
The  fifth  part  is  an  audio-slide  presen¬ 
tation  showing  the  projected  maintenance 
operations. 

PART  1.  ELECTRICAL  DESCRIPTION  OF  THE 
MX  IMU 

The  MX  IMU  consists  of  an  all-attitude, 
radiation  hardened,  floated,  stable  plat¬ 
form  that  supplies  the  MX  Guidance 
Computer  with  velocity  and  attitude  in¬ 
formation.  It  is  a  continuation  of  the 
Missile  Performance  Measurement  System 
developed  under  Air  Force  Contract  by  the 
Charles  Stark  Draper  Laboratory,  The 
program  is  in  its  fifth  year  and  four  ADP 
IMUs  have  been  built  and  delivered  to  the 
Air  Force.  These  IMUs  are  undergoing 
evaluation  and  characterization  tests. 

Figure  1.  IMU  Electronics  Block  Diagram 

This  figure  shows  the  interrelations  of 
the  functions  necessary  to  mechanize  the 
IMU. 

The  dashed  lines  represent  major  physical 
partitioning  of  the  IMU.  To  the  left  of 
the  first  dashed  line  is  the  outside 
world.  Between  the  two  dashed  lines  is 
the  external  electronics,  and  to  the 
right  of  the  second  dashed  line  is  the 
floated  stable  platform  -  (l.e,,  stable 
member).  The  stable  member  is  floated  in 
a  fluorocarbon  fluid  -  FC-77. 


POWER  FUNCTION 

•  IMU  POWER 

The  outside  world  source  of  +76  VDC 
operating  power. 

•  HI  VOLT  REG 

Contains  the  EMI  filters  and  supplies 
regulated  +68  VDC  to  the  IMU. 

•  DC /DC  CONV  +15 /REG 

Converts  the  regulated  +68  VDC  to 
secondary  operating  voltages  for  the 
external  electronics. 

•  MUXR  FILTER 

Modulates/Demodulates  data  on  the 
+68  VDC  power  line  for  transmission 
to/from  the  stable  member.  The  only 
physical  connection  to  the  stable 
member  is  via  the  power  brushes. 

•  FILTER  DC/DC  CONV  +15V  REG 

Converts  the  +68  VDC  to  secondary 
operating  voltages  for  the  stable 
member, 

TIMING  FUNCTION 

Generates  the  precision  frequencies  and 
timing  signals  for  the  IMU. 

•  HARD  TIMER 

Contains  the  Xtal  oscillator,  count¬ 
down  chain,  retention  flip-flops. 
Supplies  the  precision  frequencies. 

•  COMM 

Generates  timing  signals  in  response 
to  outside  mode  commands. 

•  TGI I  EXCIT 

Supplies  wheel  power,  pick-off  excita¬ 
tion,  and  suspension  power  to  the 
Third  Generation  T_nertial  instruments 
(TGII)  and  thrust  valves. 

STABILITY  FUNCTION 

Maintains  the  stable  member  oriented  in 
inertial  space.  This  creates  the  stable 
platform. 
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•  TGG 

The  single  degree  of  freedom  Third 
Generation  Gyros  (TGG)  supply  the 
Inertial  reference, 

•  TURBOPUMP 

The  turbopump  supplies  FC-77  under 
pressure.  The  FC-77  flow  Is  controlled 
by  the  thrust  valves  to  rotate  the 
stable  member. 

•  STAB/VALVE  ELECT 

The  outputs  from  the  TGG  Is  processed 
to  generate  control  signals  to  thrust 
valves. 

•  TORQ  GEN  DRIVE 

Under  control  of  the  outside  computer. 

It  generates  TGG  torqulng  signals. 

These  signals  are  used  for  test  and 
calibration. 

VELOCITY  FUNCTION 

Measures  velocity  along  three  orthogonal 
axis. 

•  SFIR 

Specific  ^orce  Integrating  Resolver 
(SFIR)  Is  a  precision  integrating 
accelerometer.  Its  resolver  output 
is  a  whole  angle  representing  velocity . 

•  VELOCITY  READOUT 

Converts  the  SFIR  outputs  to  digital 
words  for  transmission  to  the  outside 
world. 

ATTITUDE  FUNCTION 

Determines  the  attitude  of  the  stable 
member  with  respect  to  the  outside  world. 

•  ATTITUDE  EXCITATION 

Supplies  the  signals  that  excite  the 
driver  bands.  Three  driver  bands  are 
mounted  orthogonally  on  the  surface  of 
the  stable  member. 

•  ATT  BANDS 

The  three  driver  bands  Induce  a  signal 
in  the  receiver  band.  The  receiver 
band  is  mounted  on  the  equatorial  ring 
which  Is  fixed  to  the  missile  structure. 


The  driver  band  signals  are  magnetic¬ 
ally  coupled  to  the  receiver  band 
across  the  FC-77  flotation  fluid. 

•  ATTITUDE  READOUT  PROCESSOR 

Converts  the  receiver  band  signal  to 
a  digital  word  for  transmission  to 
the  outside  world. 

COMMUNICATIONS  FUNCTION 

Performs  the  communications  between  the 
outside  world  and  the  IMU. 

•  COMM  (right  of  second  dashed  line) 

Formats  outgoing  data  from  the  stable 
member.  The  data  consists  of  digital 
words,  pulse-position  modulated  analog 
data,  delta  modulated  temperature 
data,  and  discrete  pulses.  It  gener¬ 
ates  the  error  detecting  codes  asso¬ 
ciated  with  the  outgoing  data. 

It  decodes  the  incoming  commands, 
checks  for  transmission  errors,  and 
issues  the  commands  to  the  affected 
function, 

•  MUXR  FILTER 

Modulates/demodulates  the  data  from 
the  power  line. 

•  COMM  (between  the  dashed  lines) 

Receives  data  from  the  stable  member, 
checks  for  transmission  errors,  re¬ 
formats  the  data,  and  transmits  it  to 
the  guidance  computer.  It  receives 
commands  from  the  guidance  computer, 
checks  for  transmission  errors,  re¬ 
formats  the  commands,  adds  the  error 
detecting  codes,  and  transmits  the 
commands  to  the  stable  member. 

TEMPERATURE  CONTROL  FUNCTION 

Controls  the  temperature  of  the  FC-77 
flotation  fluid. 

•  CIRCULATOR  PUMP 

The  IMU  is  cooled  by  expanding  liquid 
freon  in  a  heat  exchanger.  The  FC-77 
flotation  fluid  is  pumped  across  the 
heat  exchanger  by  the  circulator  pump. 
The  speed  of  the  circulator  pump  is 
controlled  by  closing  a  loop  through 


the  computer.  The  temperature  of  the 
flotation  fluid  is  sensed  at  the 
turbopump  and  transmitted  to  the  com¬ 
puter.  The  computer  uses  this  tem¬ 
perature  value  to  compute  the  required 
speed  of  the  circulator  pump. 

SUSPENSION  FUNCTION 


Starting  from  the  right; 

•  cavity  cover 

•  TGG  electronics  w/hybrids 

•  TGG 

•  Power  conditioner 

•  Turboptirap  electronics 


The  suspension  function  maintains  the 
stable  member  centered  in  the  flotation 
chamber. 

•  SUSPENSION  PADS 

The  stable  member  is  neutrally  buoyant 
at  a  single  temperature  of  the  FC-77 
flotation  fluid.  The  stable  member 
is  centered  in  the  flotation  chamber 
by  pumping  FC-77  through  the  eight 
suspension  pads. 


•  Turbopump 

•  Attitude  driver  bands 

•  Attitude  receiver  band 

Figure  5.  Top  Cover  Removed 

The  photo  shows; 

•  The  external  electronics 

Figure  6.  Lower  Cover  Removed 


PART  2.  PHYSICAL  DESCRIPTION  OF  THE  IMU 

The  unique  mechanical,  thermal,  and  pack¬ 
aging  design  of  the  MX  IMU  is  shown  by 
a  series  of  15  photographs  showing  an 
IMU  being  disassembled. 

Figure  2.  MX  IMU  (ADP  Model) 

The  IMU  ready  for  installation  into  the 
guidance  and  control  drawer. 

The  photo  shows: 

•  Freon  Input/output  ports 

•  Electrical  Connectors 

•  Support  Ring 

Figure  3.  Exploded  View  of  the  IMU  Mock-Up 

•  Upper  cover 

•  External  electronics 

•  Upper  heat  exchanger  and  power  shell 

•  Stable  member 

•  Equatorial  ring 

•  Lower  power  shell 

Figure  4.  Exploded  View  Showing  Additional 
Details 

Starting  from  the  far  left: 

•  Cavity  cover 

•  SFIR  electronics  w/micro  circuit 
hybrids 


The  photo  shows: 

•  The  lower  heat  exchanger 

•  Volume  compensators 

•  Circulator  pump 

Figure  7.  Support  Ring  Removed 
The  photo  shows: 

•  Sphere  assembly 

•  Circulator  pump 

•  Freon  pressure  regulators 

•  Power  and  multiplexer  line 

Figure  8.  Upper  Heat  Exchanger  Removed 
The  photo  shows; 

•  Power  and  multiplexer  connector 

•  Upper  power  shell 

•  Heat  exchnnger/f lotation  interface 

•  Isolation  mounts 

•  Equatorial  ring  with  fluid 
passages 

•  Support  ring 

Figure  9.  Upper  Shell  Removed 
The  photo  shows : 

•  The  stable  member  in  the  flotation 
chamber 


•  SFIR 
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Figure  10.  The  Stable  Member  Removed 
From  the  Flotation  Chamber 


The  first  step  in  developing  a  test 
approach  is  to  define  the  test  objectives, 


The  photo  shows: 

•  Stable  member 

•  Attitude  bands  and  drive  electronics 

•  Thrust  valves 

•  Suspension  pads 

•  Power/muxr  brushes 

•  Balance  screws 

•  Cavity  covers 

Figure  11.  Removing  Cavity  Cover 
The  photo  shows: 

•  The  cavity  cover  being  removed 

•  The  attitude  driver  bands  removed 

Figure  12.  TGG  Cover  Removed 
The  photo  shows: 

•  The  TGG  mounted  in  the  cavity 
Figure  13.  The  TGG  Removed 

The  photo  shows: 

•  The  power  conditioner  Circuit  ^oard 
Assembly  (CBA)  with  hybrids 

Figure  14.  Power  Conditioner  CBA  Removed 

The  photo  shows: 

•  Turbopump  electronics  hybrids 
Figure  15.  SFIR  Cavity  and  Cover 

The  photo  shows: 

•  The  four  SFIR  CBA's 

•  SFIR 

Figure  16.  The  Stable  Member 

The  photo  shows  the  stable  member  float¬ 
ing  in  the  buoyancy  and  balance  tank. 

PART  3.  METHODOLOGY  BEING  USED  TO  DEVELOP 
THE  MX  IMU  TEST  METHODS  AND  TEST 
EQUIPMENT 


•  The  tests  must  verify  that  the  product 
meets  the  performance  requirements. 

•  Factory  and  maintenance  requirements 
are  similar,  but  not  identical.  When 
these  differences  are  not  addressed 
and  resolved  during  the  development 
phase  of  a  program,  the  product  design 
will  be  optimized  for  factory  build. 
This  results,  at  best,  in  a  set  of 
tests,  test  equipment,  test  methods, 
test  procedures,  etc.  for  the  factory 
and  a  whole  new  set  for  maintenance. 

At  worst,  it  results  in  a  product  that 
can  be  built  and  delivered  but  not 
maintained.  Thus,  these  differences 
become  major  drivers  in  the  life  cycle 
cost. 

Figure  18.  Performance  Requirements 

These  are  the  performance  requirements 

that  testing  can  verify  and  have  a  major 

effect  on  the  test  approach. 

•  System  accuracy  determines  the  test 
equipment  accuracy. 

•  Reliability  affects  factory  test 
methods.  For  example,  burn-in, 
parameter  drift  screening,  thermal 
cycle,  etc, 

•  Interchangeability  -  affects  test 
methods  -  test  in  next  assembly, 
parametric  tests,  hot  mock-up  etc. 

Figure  19.  Factory  Requirements 

These  are  the  major  factory  requirements 

that  the  test  approach  must  satisfy: 

•  System  accuracy  must  be  verified 
before  the  IMU  can  be  delivered, 

•  Special  tests  must  be  performed  to 
verify  reliability.  Maintenance  does 
not  perform  reliability  tests,  oper¬ 
ational  reliability  is  verified  by 
maintaining  failure  records. 

•  Faults  must  be  identified  before  a 
large  investment  is  made  in  the  item. 


•  Interchangeability  requires  that  the 
tests  be  comprehensive  enough  to 
insure  that  the  item  will  function 
with  the  other  elements  when  the  tol¬ 
erances  all  go  the  wrong  way. 

Figure  20.  Maintenance  Requirements 

These  are  the  major  maintenance  require¬ 
ments  that  the  test  approach  must  satisfy; 

•  The  system  accuracy  requirement  is  the 
same  as  the  factory. 

•  Faults  that  could  compromise  system 
performance  must  be  identified  at  the 
highest  level. 

•  High  valve  items  must  be  put  back  into 
service  as  quickly  as  possible.  This 
means  good  assemblies  are  not  used  to 
fault  Isolate  failed  assemblies. 

•  Downward  compatibility  requires  that 
the  tests  isolate  a  fault  to  the  next 
lowest  assembly. 

•  Interchangeability  requirement  is  the 
same  as  the  Factory. 

Figure  21.  Controllable  Parameters 

These  are  the  parameters  that  can  be 

varied  to  meet  the  test  objectives: 

•  Partitioning  defines  how  functions  are 
grouped  -  the  circuit  layouts,  access¬ 
ibility,  etc.  This  is  a  high  payoff 
area  sometimes  referred  to  as  design¬ 
ing  for  testability. 

•  Test  point  selection  -  the  test  points 
selected  have  a  direct  effect  on  the 
testability  of  the  item. 

•  The  test  equipment  determines  what  can 
be  measured,  how  quickly  measurements 
can  be  made,  what  information  can  be 
extracted  and  displayed.  It  also 
influences  how  quickly  and  accurately 
decisions  can  be  made. 

•  Test  methods  define  what  parameters 
are  being  measured. 

Figure  22.  Test  Development  Methodology 


being  refined. 

•  IMU  DESIGN 

The  design  of  the  IMU  is  the  principle 
driver. 

•  MAINTENANCE  PHILOSOPHY 

The  maintenance  philosophy  defines  how 
the  IMU  will  be  maintained  during  its 
operational  life. 

•  FACTORY  TEST  FLOWS 

These  show  how  the  factory  plans  to 
build  the  IMU.  They  identify  where 
tests  will  be  performed.  During  the 
build  cycle,  the  flow  is  upward. 

•  DEPOT  TEST  FLOWS 

These  show  how  the  depot  plans  to 
fault  isolate  and  repair  failed  IMUs. 
These  flows  are  both  downward  and 
upward. 

•  DECISION  POINTS 

Decision  points  are  where  tests  are 
performed  and  decisions  made.  Main¬ 
tenance  philosophy  and  test  flows  are 
modified  to  create  a  common  set  of 
decision  points.  This  is  one  step  in 
resolving  the  differences  between 
maintenance  requirements  and  factory 
requirements. 

•  TEST  REQUIREMENTS 

These  are  the  performance  requirements 
that  the  testable  items  must  meet  in 
order  that  the  IMU  will  meet  its  per¬ 
formance  requirements, 

•  TEST  EQUIPMENT  TEST  METHODS 

The  decision  points  define  what  will 
be  tested  and  the  test  requirements 
define  the  measurements  that  must  be 
made.  These  two  Inputs  have  a  major 
Influence  in  defining  the  test  equip¬ 
ment  and  test  methods. 

•  TESTABILITY  ANALYSIS 

Testability  analysis  Includes  reli¬ 
ability  predictions,  failure  and 
effects  analysis  and  maintainability 
analysis,  as  well  as  test  equipment 
capabilities,  test  methods,  and  test 
requirements.  The  output  of  the 
testability  analysis  modifies  the 


This  is  a  flow  diagram  of  how  the  tests 
for  the  MX  IMU  were  developed  and  are 


partitioning  and  test  point  selection  of 
the  IMU  design  which  iterates  the  whole 
process. 


This  is  not  a  trivial  process  -  in  the 
MX  IMU  program,  a  number  of  person  years 
of  effort  have  been  expended.  Special 
tradeoff  studies  have  been  made  to  de¬ 
termine  what  functions  go  where,  what 
signals  should  be  monitored,  how  the 
signals  should  be  monitored,  what  test 
equipment  should  be  used,  what  data 
should  be  collected,  and  how  the  data 
should  be  processed. 

As  can  be  seen  from  the  elements  Involved 
in  the  flow  diagram,  the  MX  IMU  test 
development  is  part  of  the  basic  design 
process.  When  this  is  done  correctly, 
the  result  is  a  product  that  can  be  built 
efficiently,  maintained  effectively,  and 
used  with  a  high  confidence  of  success. 


PART  A.  THE  STATUS  OF  THE  MX  IMU  TEST 
METHODS  AND  TEST  EQUIPMENT 

Figure  23.  IMU  Maintenance  Philosophy 

This  is  the  maintenance  philosophy  pres¬ 
ently  being  implemented. 

Figures  2A  and  25.  Decision  Points 

This  shows  the  points  where  tests  will 
be  performed  in  the  factory  and  in  the 
depot.  The  differences  between  the  fac¬ 
tory  and  depot  are  a  result  of  the  depot 
not  repairing  hybrids,  but  repairing 
TGIIs  and  hydraulic  components. 

Figures  26  through  35. 

These  show  the  planned  test  equipment  and 
test  methods  that  will  be  used  to  support 
the  decision  points. 

PART  5.  MAINTENANCE  OPERATIONS 

This  is  a  self-contained  audio  slide  pre¬ 
sentation  showing  the  projected  mainte¬ 
nance  operation.  It  uses  photographs  of 
ADP  equipment  to  demonstrate  the  steps 
required  to  fault  Isolate  and  repair  an 
IMU. 


•  IFSS  ICLIUILm  REWIREICIITS 

•  PARTS  SEUaiON  M.TBMTIVES 

•  COST  OF  ELECTRONIC  PARTS  VS  FAILURE  RATES 

•  PARTS  COST  TRADE  OFF  FOR  HULTIPUXER  MASTER  WIT  (NUX) 

•  PARTS  COST  TRADE  OFF  FOR  IFSS  AIRBORNE  EOUIPfCNT 

•  DURING  PREUUNCH  CHECKOUT 

•  DURING  aiGHT  TEST 

•  COniNED  PRELAUNCH  AM)  aiGHT  TEST 

•  IFSS  RELIABILITY  PERDiaiONS 

•  CONCLUSIONS 

•  RECONSOIATIONS 


LI-AUKCH  AND  FLIGHT  RELIABILITY  (1  HOUR  PRELAUNCH,  55  BIN.  FLIGHT) 

REmaiLiiY 

SS  0.980 

TELENETRY  FUNCTION  0.981 

TELEMETRY  GtC  DATA  0.999 

F5S 

TRACKING  FUNCTION  0.995 

FLIGHT  TERMINATION  0.999 


500  HOURS  MTBF  PRELAUHCH 


•  electronic  parts  are  the  principal  contributor  to  IFSS  UNRELIABILITY. 

•  PAINTS  COSTS  ARE  BASED  0“  AVERAGES  OF  1978  PRIClfiG  iriFOWATION  FROM 
flAJOR  SUPPLIERS. 

•  PARTS  FAILURE  RATES  FOR  Hl-REL  PARTS  BASED  OS  EXPERIENCE.  (AUTONETICS 
APRIL  1978) 

t  PARTS  FAILURE  RATES  FOR  rjLSPIC  PARTS  BASED  ON  MIL-H0Bt(-?17B. 

COST  OF  ELICTRONIC  PUTS  VS  FAILURE  RATES 
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I  THE  m  IS  REPRESENTATIVE  OF  IFSS/ABl  ELECTRONICS. 

f  THE  «t/X  k/LL  BE  SmiLAP  TO  THE  LGfT-50  PCr  WITIPLEXER. 

•  THE  COST  OF  THE  NUX  IS  ESTIMATED  AT  J125,000. 

•  RISK  COST  IS  DEFINED  AS  MUX  COSTS  X  PROBABILITY  OF  FAILURE 

f  TOTAL  .MUX  OPERATING  Tift  IS  200  HOURS. 
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•  MITM  COrtCRClAL  PARTS.  THE  PROBABILITY  OF  FAILURE  IS  EXCESSIVE.  MISSILE  RECYCLE 
COST  AND  MISSION  RISK  COSTS  ARE  PROHIBITIVE. 


•  WITH  TSE/MSE  PARTS.  THE  PREDICTED  RELIABILITY  FALLS  SUBSTANTIALLY  BELOU  REOUIREPE'. 


•  WITH  ASI  PARTS,  THE  RELIABILITY  «nS  SYSTEM  REfiUIR£«NTS  HHILE  TOTAL  COSTS 
ARE  NEAR  MINIMUM. 


•  MITM  AVE/OSE  PARTS,  THE  PROBABILITY  OF  FAILURE  IS  DECREASED  BY  A  FACTOR  OF  5 
OVER  THAT  OF  ABI  PARTS  AT  AN  INCREASED  COST  OF  t2SO,000  PER  SYSTEM. 


•  USE  ABI  LEVEL  PARTS  AS  CURRENTLY  SPECIFIED. 


I  REQUIRE  CONTRACTOR  TO  PROVIDE  JUSTIFICATION  FOR  ANY  PLANNED  USE  OF 
HI-REL  PARTS  IN  IFSS  HARDWARE. 


IFSS  Physical  Installation 
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PRELAUNCH  1  FLIGHT  RELIABILITY* 
PREDICTION  1  Pin  RFauupgHT 

FIELD  RELIABILITY  (MTBF) 

PREDlcnON  1  PinjEflUlKIfil 

®  IFSS 

0-970 

0.980 

70  HRS 

500  HRS 

TELEMTRY  FUNCTION 

0.974 

0.981 

telemetry  Gic  •• 

0.995 

0.999 

.  •  FSS 

1 

- 

TRACKING  FUNCTION 

0.99S 

0.995 

mTHUKEHTA  TIOH  AND  FlIBHT  SAFITY  SYSTtH 


MX  IMU  TEST  OBJECTIVES 

•  INSURE  HMU  MEETS  PERFORMANCE  REQUIREMENTS 

•  TESTS  MEET  FACTORY  REQUIREMENTS 

•  TESTS  MEET  MAINTENANCE  REQUIREMENTS 

•  MINIMIZE  LIFE  CYCLE  COST 

PERFORMANCE  REQUIREMENTS 

•  SYSTEM  ACCURACY 

•  RELIABILITY 

•  INTERCHANGEABILITY 

FACTORY  REQUIREMENTS 

•  SYSTEM  ACCURACY 

•  RELIABILITY 

•  UPWARD  COMPATIBILITY 

•  IDENTIFY  FAULTS  AT  LOWEST  LEVEL 

•  INTERCHANGEABILITY 


MAINTENANCE  REQUIREMENTS 

•  SYSTEM  ACCURACY 

•  IDENTIFY  FAULTS  AT  HIGHEST  LEVEL 

•  MINIMIZE  DOWN  TIME 

•  DOWNWARD  COMPATIBILITY 

•  INTERCHANGEABILITY 

CONTROLLABLE  PARAMETERS 

•  SYSTEM  PARTITIONING 

•  TEST  POINT  SELECTION 

•  TEST  EQUIPMENT 


•  TEST  METHODS 


TEST  DEVELOPMENT  METHODOLOGY 
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IMU  MAINTENANCE  PHILOSOPHY 


HYBRIDS 


LEVEL 

-  NO  PLANNED  IMU  MAINTENANCE 

-  NO  PLANNED  IMU  MAINTENANCE  TRANSPONTEO 
TO  DEPOT  IN  MOCS  ONAWEN 

•  REMOVE  AND  REPLACE  ASSEMBUE8  BASED  ON 
FAILURE  DATA 

FUNCTIONAL  TEST.  REMOVE  AND  REPLACE  ASSEMBLIES 
RETURN  IMU  TO  USE 

REPAIR  ASSEMBLIES  BY  REPLACINQ  FAILED  PARTS 


DECISION  POINTS 

TEST  ITEM 

FACTORY  (NED)  DEPOl 

HYtmOMATEMAL 

X 

IN-PROCESS  HYBRID 

X 

HYBRIDS 

X 

X 

TOS  MATERIAL 

X 

m-PROCESS  TON 

X 

TOtt 

X 

X 

HYDRAULIC  MATERIAL 

X 

M-PROCESS  HYDRAULIC 

X 

MECHANICAL  PARTS 

X 

X 

EUCTRICAL  PARTS 

X 

X 

CBA't 

X 

X 

ASSEMBLIES 

X 

X 

EXTERNAL  ELECTRONICS 

X 

X 

8TABU  MEMBER 

X 

X 

SPHERE  ASSEMBLY 

X 

X 

HWU 

X 

X 

IN-PROCESS  HYBRIDS 

TEST  EQUIPMENT 

COMMERCIAL  ATE 

INTERFACE  ADAPTERS 

BURN-IN  FIXTURES 

TEST  METHODS 

PARAMETRIC  (THERMAL) 

PARAMETER  DRIFT  SCREENING 

BURN-IN 

THERMAL  CYCLE 

SOFTWARE 

PARAMETRIC  TEST 

DIAGNOSTICS 

DATA  BASE 

0R0ANIZAT10NAL 

INTERMEDMTE 

DEPOT 


TEST  EQUIPMENT 


COMMERCIAL  ATE 
INTERFACE  ADAPTERS 


TEST  METHODS 


PARAMETRIC  (THERMAL) 


SOFTWARE 


PARAMETRIC  TEST 
DATA  BASE 


TO  II 


TEST  EQUIPMENT* 


TEST  METHODS 


SOFTWARE 


•  SAME  AS  MANUFACTURER 


COMMERCIAL  ATE 
INTERFACE  ADAPTERS 
PRECISION  FIXTURES 

PARAMETRIC 

PREUMINARY  CAUBRAHON 

PARAMETRIC  TEST 
PRELIMINARY  CALIBRATION 
DATA  BASE 


HYDRAULIC  ELEMENTS 


TEST  EQUIPMENT  SEMI-AUTOMATIC  (SPECIAL) 

INTERFACE  ADAPTERS 


TEST  METHODS  PARAMETRIC 

SOFTWARE  DATA  BASE 
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CBA’S 


EXTERNAL  ELECTRONICS 


TEST  EQUIPMENT 


COMMERCIAL  ATE  TEST  EQUIPMENT 

INTERFACE  ADAPTERS 


COMMERCIAL  ATE 
INTERFACE  ADAPTER 


TEST  METHODS 


PARAMETRIC 
THERMAL  CYCLE 


SOFTWARE  PARAMETRIC  TEST 

DIAGNOSTICS* 
DATA  BASE 

•  FAULT  ISOLATE  TO  PART  WITHOUT  PROBING 


TEST  METHODS  PARAMETRIC 

SOFTWARE  PARAMETRIC  TEST 

DIAGNOSTIC* 
DATA  BASE 

*  FAULT  ISOLATE  TO  NEXT  LOWER  ASSEMBLY 


STABLE  MEMBER 


ASSEMBLIES 


TEST  EQUIPMENT 


TEST  EQUIPMENT  COMMERCIAL  ATE 

SEMI-AUTOMATIC  (SPECIALI* 

INTERFACE  ADAPTERS  METHODS 


COMPUTER 
INTERFACE  ADAPTER 
ELECTRONIC  SIMULATOR 
HOLDING  FIXTURE 

PARAMETRIC 
SECOND  CALIBRATION 


TEST  METHODS  PARAMETRIC 

SOFTWARE  PARAMETRIC  TEST 

DIAGNOSTICS** 
DATA  BASE 


*  HYDRAULIC  ASSEMBLIES 


SOFTWARE  PARAMETRIC 

CAUBRATION 
DIAGNOSTICS* 
DATA  BASE 


*  FAULT  ISOLATE  TO  CAVITY.  THRUST  VALVE  ASSEMBLY  OR  ATTITUDE  BAND. 


*  FAULT  ISOLATE  TO  THE  NEXT  LOWER  ASSEMBLY 


SPHERE  ASSEMBLY 


TEST  EQUIPMENT 


TEST  METHODS 


SOFTWARE 


*  SAME  AS  STABLE  MEMBER 


COMPUTER* 

INTERFACE  ADAPTER* 
ELECTRONIC  SIMULATOR* 
HOLDING  FIXTURE 
THERMAL  CONTROL 

PARAMETRIC* 

SECOND  CALIBRATION* 
VERIFICATION 

PARAMETRIC 

CALIBRATION 

VERIFICATION 

DIAGNOSTICS* 

DATA  BASE 


IMU 


TEST  EQUIPMENT  COMPUTER* 

INTERFACE  ADAPTER* 
HOLoma  nxTURES 
SHAKER 
REFRIOERANT 

TEST  METHODS  PARAMETRIC 

FINAL  CAUBRATION 
VIBRATION 

SOFTWARE  PARAMETRIC 

CALIBRATION 
VIBRATION 
DMONOSTICS** 

DATABASE 


*  SAME  AS  STABLE  MEMBER 

••  FAULT  IBOLATION  TO  EXTERNAL  ELECTRONICS.  SPHERE  ASSEMBLY. 
STABU  MEMBER.  CAVITY.  THRUST  VALVE  ASSEMBLY.  OR  AnTTUOE 
BAND  ASSEMBLY. 


PROVISIONS  FOR  NORMALLY  POWER-OFF 
(DORMANT)  ELEMENTS  OF  ICBM  SYSTEMS 
TO  ASSURE  OPERATIONAL  PERFORMANCE 

by 

Loren  D.  Dierking  and 
Robert  F.  Nease 

Autonetics  Strategic  Sy^ems  Division 

Rockwell  iitemational 

BACKGROUND 

The  brief  study  referenced  herein  was  under¬ 
taken  during  the  early  R&D  phase  of  the  MX 
missile  development  program.  The  study  per¬ 
tains  to  a  segment  of  the  guidance  and  control 
(G&C)  electronics  which  was  to  be  mounted 
"downstage"  —  in  this  particular  case,  in  a 
box  mounted  on  Stage  I  and  in  a  box  mounted 
on  Stage  n.  These  electronics,  called  Servo 
Amplifier  Assemblies  (SAAs),  are  depicted 
by  cross-hatched  blocks  in  Figure  1. 

The  SAAs  provide  the  rather  simple  function 
of  interfacing  between  the  "upstage"  G&C  elec¬ 
tronics  and  the  pitch  and  yaw  turbo-mechanical 
thrust  vector  actuators  (TVAs)  which  position 
the  single  moveable  nozzles  employed  by 
Stages  I  and  II.  The  "drive"  signals  from  the 
upstage  G&C  are  in  the  form  of  low-level 
fixed  amplitude  pulsewidth-modulated  sig¬ 
nals,  and  these  signals  need  only  be  converted 
to  higher  power  levels  to  operate  the  clutch 
drive  motors  of  the  TVAs.  The  principal  rea¬ 
sons  for  locating  these  SAA  electronics  down¬ 
stage  is  to  avoid  high  current  flow  in  long  seg¬ 
ments  of  cabling  and  to  take  advantage  of  the 
payload  efficiency  of  providing  power  from 
downstage  sources  rather  than  from  upstage. 

In  regard  to  this  latter,  the  SAAs  derive  their 
primary  power  by  rectifying  the  output  of  a 
three-^ase  alternator  which  is  a  part  of  the 
TV  A  system.  Also,  the  SAAs  provide  a  simple 
speed- control  function  for  the  hot-ggs  turbine 
drive  of  the  TVAs.  These  SAA  functions  are 
depicted  in  the  block  diagram  of  Figure  2. 
Preliminary  design  studies  indicate  that  each 
of  these  SAAs  would  have  approximately  800 
electronic  parts,  many  with  high  current 
ratings. 

As  a  result  of  physical  constraints  of  the 
installation  and  other  application  matters,  the 
turbo-mechanical  TVAs  cannot  be  tested,  or 
operated,  after  the  stage  has  been  assembled 
into  the  missile  -  until  their  operation  during 
booster  flight.  Thus,  unless  special  provisions 
are  made  specifically  for  this  purpose,  the 


SAAs  cannot  be  tested,  or  even  powered, 
during  the  pre-launch  period  of  weapon  sys¬ 
tem  operation.  These  various  operating  con¬ 
cepts  and  constraints  resulted  in  the  formula¬ 
tion  of  a  rather  well-defined  trade  study  to 
determine  whether  or  not  to  include  provi¬ 
sions  for  pre-launch  testability  in  the  design 
of  the  SAAs.  Since  ground-based  power  and 
cooling  constraints  precluded  the  considera¬ 
tion  of  continuous  pre-launch  operation  of  the 
SAAs,  the  trade  study  was  limited  to  consid¬ 
eration  of  two  basic  approaches: 

1.  Dormant  SAAs  —  that  is,  SAAs  which 
are  neither  powered  nor  tested  after 
missile  assembly  until  the  missile  is 
launched,  or 

2.  Semi-Dormant  SAAs  —  that  is,  SAAs 
which  can  be  powered  up  and  tested 
periodically  during  ground  operations 
of  the  missile.  This  design  approach 
is  also  referred  to  as  the  Testable 
SAA. 

Although  dormant  and  semi-dormant  design 
approaches  could  have  covered  a  wide  range 
of  designs/capabilities,  it  will  be  noted  in  the 
following  section  that  only  one  design  concept 
was  considered  seriously  for  each  of  these 
approaches.  Before  proceeding,  it  is  also 
worthy  of  note  that  there  was  probably  some 
significant  prejudice  existing  -  both  with  this 
contractor  and  our  customer  -  in  favor  of  the 
semi-dormant  approach,  since  this  was  the 
approach  used  on  all  three  Minuteman  pro¬ 
grams  for  downstage  mounted  flight  control 
electronics.  But  more  about  this  in  a  later 
section. 

SAA  TESTABILITY  TRADES 

In  addition  to  cost,  size,  weight,  and  other 
obvious  parameters  which  were  to  be  com¬ 
pared  in  this  trade  of  test  vs  no-test  (or  semi- 
dormant  vs  dormant)  SAA  designs,  there  are 
three  requirements/constraints  on  the  G&C 
system  (of  which  the  SAAs  are  a  part)  which 
could  be  significantly  influenced  by  SAA  per¬ 
formance.  These  are: 

1.  Pg.  The  G&C  system  must  have  a 
probability  of  successful  flight  and 
launch  (called  Pg)  which  exceeds  a 
specified  value.  That  is,  whichever 
SAA  approach  is  selected  must  pro¬ 
vide  a  box-level  Pg  which  is  consist¬ 
ent  with  this  G&C  system  require¬ 
ment.  Although  not  required  by  con¬ 
tract,  a  tentative  Ps  allocation  for 
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Similarly,  using  postulated  failure  rates  for 
dormant  components,  the  MIL  Handbook-type 
procedures  were  used  to  estimate  Pg  for  toe 
dormant  version  of  the  SAA.  In  this  case,  it 
was  found  that  Ps  for  the  dormant  SAA  was 
marginally  acceptable  after  several  years  of 
deployment.  However,  the  expected  "matura¬ 
tion"  of  the  Ps  for  the  two  different  versions 
of  the  SAAs  was  considered  to  be  distinctly 
different,  as  shown  conceptually  in  Figures  6 
and  7.  More  specifically,  it  is  expected  that 
the  greater  exposure  experienced  with  the 
testable  SAA  during  R&D  and  early  operational 
use  will  allow  the  "bugs"  to  be  removed  from 
the  testable  SAA  hardware  more  quickly  than 
they  will  be  removed  from  toe  dormant  SAA 
hardware.  Also,  Figure  7  shows  the  gradual 
reduetion  in  Pg  with  passage  of  time  which  can 
be  expected  with  the  dormant  SAA. 


In  summary,  it  was  concluded  that  either  ap¬ 
proach  could  be  made  to  work,  with  costs 
favoring  the  dormant  SAA  concept,  and  pre¬ 
dicted  reliability  performance  favoring  the 
testable  SAA  concept.  The  testable  SAA  ap¬ 
proach  was  favored  for  toe  following  reasons: 

•  Hardware  maturity  is  likely  to  be 
achieved  earlier  in  the  program. 

•  Surprises  and  disappointments  are  less 
likely. 

•  Generally  improved  confidence  in  hard¬ 
ware  performance  can  be  obtained 
with  only  modest  increase  in  cost. 


Figure  1.  Missile  Flight  Control  Subsystem 


Figure  3.  Testable  SAA  Block  Diagram 


Figure  2.  Dormant  SAA  Block  Diagram 


Figure  4.  Missile  F/C  Hazard  Rate  at 
Missile  Assembly 


the  SAAs  had  been  established  by  the 
contractor. 

2.  MTBF.  The  G&C  system  must 
achieve  a  pre-launch  reliability  of  at 
least  a  specified  value  for  meantime 
between  failure  (or  MTBF).  Again, 

a  tentative  allocation  of  this  system- 
level  requirement  to  the  SAA  was 
available  for  the  trade  studies.  Obvi¬ 
ously,  this  reliability  requirement  is 
pertinent  only  to  the  testable  (or 
semi-dormant)  version  of  the  SAA, 
since  the  dormant  SAA  cannot  fail 
during  pre-launch  missile  operations. 

3.  Fault  Isolation.  G&C  faults  detected 
during  pre-launch  operations  must  be 
isolated  to  the  faulty  line- replaceable 
unit  (LRU)  through  monitoring,  self¬ 
test,  and  status  reporting,  with 
positive  identification  of  95  percent 
of  all  failures.  As  with  the  MTBF 
requirement,  this  requirement  is 
important  only  to  the  semi-dormant 
version  of  the  SAA. 

After  a  brief  review  of  the  two  general  ver¬ 
sions  of  the  SAAs,  it  was  determined  that  the 
two  concepts  could  be  rather  well  defined  as 
follows: 

•  The  dormant  (or  non-testable)  version 
was  as  defined  in  the  block  diagram  of 
F igure  2  -  without  the  addition  of  re¬ 
dundant  elements,  due  to  weight/size 
limitations. 

•  The  semi-dormant  (or  testable)  ver¬ 
sion  was  defined  such  that  almost  all 
(approximately  95  percent)  of  the  SAA 
functions  could  be  tested.  The  block 
diagram  for  this  semi-dormant  ver¬ 
sion  of  the  SAA  is  shown  in  Figure  3. 

Thus,  these  two  specific  concepts  for  the 
dormant  and  semi-dormant  SAAs  represent 
what  were  considered  to  be  extremes  for  the 
two  approaches.  Also,  the  major  differences/ 
changes  in  the  programs  for  the  two  versions 
of  SAAs  were  defined  as  described  below. 

Unique  Aspects  of  Dormant  SAA  Program 

•  Additional  acceptance  tests  for  dormant 
SAAs  were  defined  to  provide  increased 
exposure  of  this  hardware  prior  to 
delivery. 


•  A  reliability  surveillance  program  was 
added  to  provide  post-delivery  moni¬ 
toring  of  this  dormant  hardware.  This 
program  involved  the  testing  of  10  or 
more  "aged"  SAAs  each  year.  B;  was 
considered  that  this  testing  would  be 
non-destructive  and  that  each  test 
specimen  could  be  returned  to  opera¬ 
tional  inventory  after  the  sample 
testing. 

•  The  SAAs  probability  of  successful 
launch  and  flight  (Pg)  tends  to  degrade 
with  time  after  their  installation  in  the 
missile,  due  to  presumed  failure  rate 
of  parts  under  power-off  conditions. 
This  Pg  could  be  "reset"  by  the  re¬ 
cycle  of  deployed  missiles  and  change- 
out/test  of  SAAs,  but  at  significant 
expense. 


•  Approximately  95  percent  of  the  SAA 
function  is  verified  in  periodic  (say, 
monthly)  missile  test.  This  test  pro¬ 
vision  added  approximately  15  percent 
to  the  electronics  parts  count,  and 
only  a  small  increase  in  size/weight  of 
the  box.  This  provision  for  testing  also 
added  approximately  20  percent  to  the 
wire  count  of  the  missile  cables  and 
added  a  ground  power  supply  for  such 
testing. 

•  As  a  result  of  periodic  testing,  the 
failure  of  an  SAA  can  be  detected  and 
the  failed  SAAs  must  be  replaced,  at 
significant  expense.  Such  failures  must 
be  included  in  the  allowable  G&C 
MTBF,  Also,  any  detected  faults  must 
be  properly  isolated  between  the  up¬ 
stage  and  downstage  elements,  and 
they  had  to  be  accounted  for  in  the  def¬ 
inition  of  the  test  circuitry  for  the 
testable  SAA. 

•  This  periodic  testing  (and  changeout  of 
failed  SAAs)  should  give  rise  to  a  Ps 
for  the  testable  SAA  which  is  consider¬ 
ably  higher  than  that  for  a  dormant 
SAA  after  prolonged  periods  of 
deployment. 

This  somewhat  restricted  definition  of  the  two 
competing  concepts  allowed  a  rather  simple 
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comparison  of  the  two  approaches  without  get¬ 
ting  bogged  down  in  the  myriad  versions  of 
each  concept  which  might  have  been  consid¬ 
ered.  Further  refinements  in  the  concepts 
(such  as  the  addition  of  some  redundancy  for 
the  dormant  scheme)  would  be  considered 
later  if  either  concept  did  not  meet  the  top- 
level  requirements/constraints. 

REVIEW  OF  MINUTEMAN  HI  EXPERIENCE 

Before  proceeding  with  the  trade  study  com¬ 
parisons,  it  was  considered  appropriate  to 
reflect  upon  the  experience  occasioned  by  the 
development  and  deployment  of  the  Minuteman 
in  downstage  flight  control  hardware.  This 
data  base  was  selected  for  two  pr  imary  rea¬ 
sons:  1)  the  reliability  performance  of  this 
hardware  was  judged  to  be  representative  of 
that  for  a  program  which  uses  hi^-quality 
parts  and  employs  an  effective  corrective  ac¬ 
tion  program,  and  2)  the  failure  removal  data 
were  considered  to  be  complete  and  readily 
available.  Before  presenting  the  data,  it 
should  be  noted  that  this  assessment  included 
the  TVA  hardware  and  associated  downstage 
electronics  for  all  three  stages  of  the  booster. 
Thus,  the  complexity  of  this  hardware  is  sev¬ 
eral  times  that  of  the  SAA  box  under  considera¬ 
tion.  Also,  it  should  be  noted  that  all  of  this 
hardware,  including  TVAs,  is  thoroughly 
tested  during  missile  assembly  and  during 
periodic  missile  test  in  silo  operation. 

The  first  data  which  were  reviewed  pertained 
to  the  testing  of  this  flight  control  hardware 
during  initial  assembly  of  the  missile  at 
Plant  77,  located  at  Hill  Air  Force  Base.  Dur¬ 
ing  eight  years  of  missile  assembly  work,  it 
was  found  that  the  downstage  flight  control 
hardware  experienced  approximately  one  fail¬ 
ure  for  every  12.5  missiles  assembled,  or  a 
hazard  rate  for  the  downstage  fli^t  control 
hardware  of  approximately  8  percent  per 
missile. 

As  this  hazard  rate  was  broken  down  by  cal¬ 
endar  years,  it  was  found  that  some  improve¬ 
ment  was  experienced  as  the  program  ma¬ 
tured,  as  shown  in  Figure  4;  however,  even 
in  the  later  years  of  the  program,  the  hazard 
rate  was  still  significant.  Since  a  failure  rate 
of  downstage  flight  control  hardware  on  the 
order  of  8  percent  would  be  unacceptable  for  a 
totally  dormant  flight  control  system,  it  is 
easily  seen  that  additional  pre-delivery  test/ 
inspection  procedures  would  have  been  re¬ 
quired  to  make  the  Minuteman  IH  flight  control 
hardware  operate  acceptably  in  a  dormant 
application.  This  siqjported  our  initial 


thou^ts  that  a  successful  dormant  SSA  pro¬ 
gram  must  involve  more  thorough  pre¬ 
delivery  testing  than  would  be  required  for  a 
successful  testable  SAA  program. 

The  second  set  of  Minuteman  in  data  reviewed 
was  the  in- silo  failure  rate  data  for  the  down¬ 
stage  flight  control  hardware.  It  was  found 
that  this  hardware  in  the  first  seven  years  of 
silo  deplojnnent  failed  at  an  average  rate  of 
approximately  1. 5  percent  of  the  deployed 
missiles  jJer  year  of  operation.  In  this  case, 
the  breakdown  of  failure  rate  by  years  showed 
a  more  interesting  situation,  as  shown  in  Fig¬ 
ure  5.  The  failure  rate  in  the  first  three 
years  of  silo  use  was  significantly  greater  than 
that  for  subsequent  use.  This  experience  was 
directly  related  to  two  time-dependent  failure 
modes  which  were  not  experienced  during  the 
R&D  program,  but  which  were  then  identified 
and  fixed  during  the  production  program.  This 
sort  of  experience  provided  emphasis  on  the 
reliability  surveillance  effort,  which  was  also 
considered  to  be  necessary  for  a  successful 
dormant  SAA  program. 

RESULTS  OF  SAA  TESTABILITY  TRADES 

The  major  results  of  the  SAA  testability 
trades  are  summarized  below. 

Comparison  of  Program  Costs 

Overall  program  costs  for  SAA- related  ef¬ 
forts  were  found  to  be  from  5  to  10  percent 
higher  for  the  testable  SAA  than  for  the  dor¬ 
mant  SAA.  The  major  portions  of  these  added 
costs  were  attributed  to  deliverable  SAA  hard¬ 
ware  itself,  both  during  R&D  and  the  produc¬ 
tion  phases  of  the  program.  Support  costs 
attributable  to  the  testable  features  of  this 
version  of  the  SAA  were  found  to  be  a  minor 
effect.  The  cost  of  the  reliability  surveillance 
efforts  for  the  dormant  SAA  program  was 
significant,  but  it  did  not  completely  offset  the 
costs  of  the  added  complexity  of  the  testable 
SAAs.  S  this  sample  testing  had  been  assumed 
to  be  destructive,  the  cost  of  the  dormant  SAA 
program  would  have  exceeded  that  for  the  test¬ 
able  SAA  program. 


Comparison  of  Reliability  Performance 


MEL  Handbook  procedures  were  used  to  esti¬ 
mate  MTBF  and  Pg  for  the  testable  version  of 
the  SAA.  The  predicted  MTBF  exceeded  that 
which  could  be  allocated  to  the  SAA,  and  like¬ 
wise  the  predicted  Ps  exceeded  that  which 
could  be  reasonably  allocated  to  the  SAA. 
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Figure  7.  Expected  Maturation  of 
Dormant  SAA 


Figure  5.  Missile  F/C  Silo  Failure  Rate 
vs  Time 
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Figure  6.  Expected  Maturation  of 
Testable  SAA 


DEVELOPMENT  PROCESS 
SUMMARY  OF 

ISSUES  AND  RECOMMENDATIONS 

1.  Over-lapping  Specifications 
Issue 

Closely  similar  specifications  in¬ 
dicate  a  need  for  commonality  or 
standardization  of  existing  spec¬ 
ifications  to  limit  development 
costs.  As  a  by-product  fo  the  cost 
and  schedule  sensitivities  brought 
out  in  workshop  discussions,  the 
issue  of  proliferation  and  confus¬ 
ion  of  well-intended  specifications 
was  raised.  In  particular,  partici¬ 
pants  were  concerned  over  electro¬ 
nic  piece-part  specifications  and 
soldering  (craftsmanship)  specifi¬ 
cations  residual  to  earlier  space 
projects.  Many  of  these  have  be¬ 
come  the  agency  or  company 
standards  of  various  system 
organizations  in  Government  and 
Industry.  Such  standards  are 
needed  as  instructions  at  some 
lower-tier  levels.  In  others  and 
at  major  system  contractor's 
plants,  they  are  matters  of  cost 
and  schedule  concern  for  no-longer 
-needed  variety. 

Recommendations 

o  Industry  associations  identify  to 
Government  the  specifications 
that  have  over-lapping  require¬ 
ments  and/or  redundant  require¬ 
ments. 

o  Government  agencies  and  industry 
associations  intensify  their  efforts 
to  achieve  commonality  of  spec¬ 
ifications  used  for  procurement. 

2.  Value  Analysis 
Issue 

Value  analyze  system  and  mission 
assurance  requirements,  partic¬ 
ularly  in  the  early  program  (i.  e.  , 
conceptual  and  development). 


Discussion  from  workshop  partici¬ 
pants  supported  the  general  con¬ 
cept  of  value  analysis  of  assurance 
requirements  early  in  development 
planning.  Discussion  of  a  cost 
minimization  study  leading  to 
choice  and  specifications  of  grade 
of  electronic  parts  showed  chall¬ 
enge  of  complete  life  cycle  cost 
consequences  of  such  choices. 
Concern  was  expressed  that  there 
were  hidden  development  process 
costs  consequent  on  use  of  less 
than  highest  grade  parts. 

Recommendations 

o  Require  early  reviews  of  system 
and  mission  assurance  require¬ 
ments  and  budgets. 

o  Tailor  specifications  in  the 
initial  program  phase  to  assure 
cost  effective  application  to  real 
program  requirements. 

3.  Prototype  Concept 

Issue 

Protoflight  development  concept. 
Both  Government  and  Industry 
representatives  contributed  to 
recognition  that  at  least  four  NASA 
Centers  and  two  system  prime  con¬ 
tractors  have  used  the  protoflight 
approach  and  that  it  has  represent¬ 
ed  planning  initiative  from  each  of 
Government  and  Industry. 

Recommendations 

o  Programs  consider  in  conceptual 
phase  the  use  of  a  single  set  of 
hardware  for  ground  tests  and 
flights. 

o  Make  risk  assessment, 
o  Consider  weight  and  power 
margins. 

o  Take  advantage  of  known 
mission  environment, 
o  Utilize  only  experienced  con¬ 
tractors. 

o  Whe  never  possible,  select 
proven  flight  hardware. 


WORKSHOP  I 

COMPUTER  SOFTWARE/COMPUTER  RESOURCES 


Chairmen  Coordinator 

Lt.  Col  Ed  Myers,  Space  Division  Bob  Berri,  Aerospace  Corp. 

Mr.  Dan  Schneider,  TRW 
Mr.  David  Aichele,  NASA 


Software  Management  Methods 
Plant  Representative  Methods 
Core  Concept 
Thursday,  May  1 
Software  Quality  Assurance 
Verification  and  Validation 
Air  Force  Planning 

Working  Lunch,  Open  Discussion,  Development 
of  Recommendations 


Mr.  Bob  Williams,  TRW 
Capt  George  T revor,  AFPRO 
Maj  Jim  Riley,  ALD 

Mr.  Dick  Evans,  4C  Corp 
A.J.  Macina 

Capt  Dave  Herriko,  AFSC 


SOFTWARE  MANAGEMENT  METHODS 

R.D.Williams 

TRW 

F.S.  Ingrassia 
TRW 


MILESTONES  AND  REVIEWS 


ISSUE:  SHOULD  SOFTWARE  DEVELOPMENT  METHODOLOGY  BE  A 
CONTRACT  COMPLIANCE  ITEM? 


WHAT  IS  SOFTWARE  DEVELOPMENT  METHODOLOGY? 


•  AN  INTEGRATED  AND  CONSISTENT  SET  OF  POLICIES,  PROCEDURES  AND  PRACTICES 
SPECIFYING  HOW  A  SOFTWARE  PRODUCT  WILL  BE  SPECIFIED,  DESIGNED,  TESTED, 
CONTROLLED  AND  DELIVERED 

/  T  INCL  UDES 

•  MILESTONES  AND  REVIEWS 

•  CORL'S  AND  DOCUMENTATION  REQUIREMENTS 

•  GOVERNMENT  COMPLIANCE  DOCUMENTS 

•  DEVELOPMENT  PRACTICES 

•  TESTING  REQUIREMENTS 

•  CONFIGURATION  MANAGEMENT 

•  QUALITY  ASSURANCE 

•  PROGRAMMING  STANDARDS 


•  THESE  CREATE  THE  BASIC  STRUCTURE  FOR  THE  DEVELOPMENT 
PROCESS 

•  NATURAL  PROGRESSION  AND  INTER  OEPENOENCY  OF  KEY 
EVENTS 

-  SRR,  SDR,  PDR,  COR,  ETC. 

•  THE  SCHEDULE  AND  CONTENT  SHOULD  REFLECT  THE  TYPE  OF 
PROCUREMENT  AND  DEVELOPMENT  EFFORT  INVOLVED 

PROBLEMS  - 

•  THE  NUMBER  AND  SCHEDULE  OF  THESE  EVENTS  NOT  ALWAYS 
REALISTIC 

-  LIKE  HAVING  A  PDR  TOO  EARLY 

•  IF  YOU  DONT  HAVE  THE  REQUIREMENTS  BASELINED  •  YOU  CANT 
PROCEED  EFFECTIVELY 

•  DISAGREEMENTS  AND  CONFLICTS  ARISE  WHEN  THE  OBJECTIVES, 
CONTENTS  AND  CONDUCT  OF  THESE  EVENTS  ARE  NOT  UNDER¬ 
STOOD  OR  AGREED  TO  BY  THE  PARTIES  INVOLVED 

•  SOFTWARE  DESIGN  IS  A  CLASS  II  TYPE  ACTIVITY  -  NOT  CLASS  I 

--  CAN'T  GET  CUSTOMER  APPROVAL  FOR  EVERY 
DESIGN  CHANGE 


WHY  IS  THIS  QUESTION  AN  ISSUE? 


CDRU'S  AND  DOCUMENTATION  REQUIREMENTS 


•  UNFORTUNATELY,  RFP  S  AND  CONTRACTS  NOT  ONLY  SPECIFY  WHAT  QN6  SHOULD 
DO  BUT  TO  A  GREAT  EXTENT.  ONE  SHOULD  OO  IT 

•  THE  TREND  SEEMS  TO  BE  TOWARD  GREATER  COMPLIANCE  SPECIFICATION  RATHER 
THAN  LESS 

•  BASED  ON  HIGH  LEVEL  GOVERNMENT  POLICIES  THERE  HAS  BEEN  A  PROLIFERATION 
OF  IMPLEMENTING  DIRECTIVES.  LOCALLY  OEVELOPEO,  THAT  ARE: 

'  BASED  ON  ORGANIZATION'S  OR  EVEN  INOIVIOUALS'S  MOST  RECENT 
EXPERIENCE  BE  IT  A  SUCCESS  OR  FAILURE 

-  BASED  ON  A  SPECIFIC  EXPERIENCE  WHICH  IS  GENERALIZED 

-  INDEPENDENT  OF  THE  CONTRACTUAL  INCENTIVE  STRUCTURE 

-  BASE  DON  THE  ASSUWPTlCN  THAT  IF  A  CONTRACTOR  IS  NOT 
RIGOROUSLY  CONTROLLED,  HE  WILL  NOT  DELIVER  A  QUALITY 
PRODUCT 


SECONDARY  ISSUE 


•  THE  PROBLEM  OF  CONSISTENCY.  RELEVANCE  AND  APPLICABILITY  OF  CURRENT 
COMPLIANCE  ITEMS  IN  REP’S 

-  ANYONE  WHO  GETS  'KVOLVED  IN  RESPONDING  TO  GOVERNMENT 
RFP'S  QUICKLY  DISCOVERS  THAT  THEY  ARE  ALL  DIFFERENT 

-  EACH  ONE  APPEARS  TO  RE  A  FAIRLY  UNlOUF  PRODUCT  COMPILED 
BY  A  UNIQUE  CROUP  OF  PEOPLE 

-  IF  ONE  COULD  ATTR.BUTE  THESE  DIFFERENCES  TO  TAILORING 
FOR  THE  SPECIFIC  PPQCUREMENTS.  THIS  WOULD  NOT  PRESENT 
A  PROBLEM 


•  THESE  GIVE  SUBSTANCE  TO  THE  DEVELOPMENT  PROCESS 

•  THEY  PROVIDE  BOTH  CONTRACTUAL  AGREEMENTS  (BASELINES) 
AND  MANAGEMENT  VISIBILITY  OF  THE  DEVELOPMENT  PROCESS 

•  THE  TYPE.  NUMBER  AND  CONTENT  NEED  TO  REFLECT  THE  TYPE 
OF  PROCUREMENT  AND  DEVELOPMENT  EFFORT  INVOLVED 

PROBLEMS  - 

•  THE  NUMBER.  ALLOCATION  AND  DELIVERY  SCHEDULES  NOT 
ALWAYS  CONSISTENT  WITH  GOOD  DEVELOPMENT  APPROACH 

-  TOO  MANY  OR  WRONG  CHOICE  OF  CPCI’S 

•  DELIVERABLES  OFTEN  TIED  TO  EVENTS  THAT  ARE  NOT  POINT 
EVENTS 

-  CONFUSION  AND  MISUNDERSTANDING  ABOUT 
DUE  DATES 

•  ASSOCIATED  DATA  ITEM  DESCRIPTIONS  OFTEN  INAPPROPRIATE 
OR  INADEQUATE  TO  DESCRIBE  DOCUMENTATION 

-  EITHER  TOO  GENERAL.  OR  MUCH  TOO  DETAILED 
AND  SPECIFIC 

•  TOO  MUCH  EMPHASIS  ON  THE  PAPER  ITSELF  AND  NOT  ENOUGH 
ON  THE  CONTENTS  OF  THE  PAPER 


-  HOWEVER.  THE  IMPRESSION  THAT  ONE  GETS  IS  THAT  THE  DIFFERENCES 
ARE  A  RESULT  OF  THE  COLLECTIVE  EXPERIENCES  AND  BACKGROUND 
OF  THE  INDIVIDUALS  WHO  PREPARED  THE  DOCUMENTS 


COMPLIANCE  DOCUMENTS 


•  THESE  SPECIFICALLY  ADDRESS  THE  HOW'  OF  DEVELOPMENT 
METHODOLOGY 

•  SOME  AKE  VERYGENb^  AL  IN  NATURE  CAN  APPLY  TO  A  WEAPON 
SYSTEM  OR  A  COMPUTER  PROGRAM 

•  SOME  ARE  VERY  SPECIFIC  AND  PERHAPS  TOO  DETAILED  IN  CONTENT 

•  TREND  SEEMS  TO  BE  TOWARD  MORE  CONSTRAINTS  RATHER  THAN 
LESS 


•  VERY  LITTLE  CONSIOERAT.ON  OR  ENCOURAGEMENT  IS  GIVEN  TO 
TAILORING  FOR  THE  SPECIFIC  APPLICATION  OR  PROCUREMENT 

•  THEY  IMPOSE  STANDARDS  AND  PRACTICES  THAT  ARE  NOT  UNIVERSALLY 
TRUE  OR  VALID  THAT  SHOULD  BE  AT  THE  DISCRETION  OR  CONTROL  OF 
THE  CONTRACTOR 

•  NOT  IN  TUNE  WITH  EVOLVING  PRACTICES  AND  NEEDS  OF  SOFTWARE 
DEVELOPMENT/MANAOEMENT 

•  CONTRACTORS  BEING  FORCED  OUT  OF  THEIR  NORMAL  DEVELOPMENT 
MODE 


CONTRACTOR  SOFTWARE  DEVELOPMENT  POLICIES 


•  THESE  ADDRESS  THE  ir:TERNAL  'WHAT'  AND  'HOW*  Of  DEVEtOPMENT 
METHODOLOGY 

•  THEY  MAY  VARY  WIDELY  IN  THEIR  QUALITY  AND  DETAIL 

•  THEY  DO  REPRESENT  THE  CONTRACTOR'S  MODE  Of  OPERATION  AND 
MANAGEMENT 


•  CONFLICT  SITUATION  ARISES  WHEN  COMPLIANCE  ITEMS  ARE  IN 
DISAGREEMENT  WITH  CONTRACTOR  POLICIES  AND  PROCEDURES 

•  THE  EXTREME  VARIABILITY  AMONGST  CONTRACTORS  MAKES 
COMPARATIVE  EVALUATION  DIFFICULT 

•  WHEN  THEY  DON'T  EXIST  CHAOS  MAY  RESULT  DUE  TO  INCONSISTENT 
IMPLEMENTATION  ACROSS  A  PROJECT 

•  THEY  SOMETIMES  DON'T  HAVE  THE  FLEXIBILITY  TO  FIT  THE 
PROCUREMENT 


Respmsi  Totssui 

NO  SOFTWARE  DEVELOPMENT  METHODOLOGY  CONSTRAINTS  SHOULD 
NOT  BE  CONTRACT  COMPLIANCE  ITEMS.  AND  SHOULD  BE  REMOVED 
FROM  SPECIFICATIONS.  STANDARDS  AND  DIRECTIVES  WHERE  THEY 
OCCUR 


THE  TIME  HAS  COME  FOR  THE  OOVERNMENT/INOUSTRY  TO  TREAT 
SOFTWARE  DEVELOPMENT  METHODOLOGY  ISSUES  AS  CRITERIA  BY 
WHICH  TO  EVALUATE  THE  CONTRACTOR'S  CULTURALLY  DEVELOPED 
SYSTEM.  AND  NOT  AS  REQUIREMENTS  IN  COMPLIANCE  DOCUMENTS 


RECOMMENDATIONS 


1.  THE  GOVERNMENT  SHOULD  DISSEMINATE  GUIDELINES  AND  STANDARDS  FOR 
THE  PREPARATION  OF  RFP'<^  AND  INSTITUTE  TRAINING  PROGRAMS  IN  THEIR 
USE. 

2.  FORM  A  CENTRAL  GROUP  IN  EACH  DEPARTMENT  OR  AGENCY  TO  REVIEW  RFP'S 
FOR  CONSISTENCY.  ADEQUACY  AND  COMPLIANCE  WITH  GUIDELINES. 

3.  LIMIT  COMPLIANCE  ITEMS  TO  THOSE  THINGS  THAT  ARE  REALLY  ESSENTIAL. 

4.  IN  LIEU  OF  13)  •  SOLICIT  AND  ENCOURAGE  TAILORING  TO  FIT  THE  PROCUREMENT. 

5.  ENCCMJRAGE  CONTRACTORS  TO  DOCUMENT  AND  PROPOSE  THEIR  DEVELOPMENT 
METHODOLOGY  AND  BE  EVALUATED  ON  IT. 

B.  AS^RE  ABSENCE  OF  COMPLIANCE  ITEMS  WHICH  AOORtSS  "HOW  TO  00  IT" 
INSTEAD  OF  "WHA  T  IS  NEEDED" 
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CMSEP  OVERVIEW 


COMPUTER  SOFTWARE  CONTRACT  ADMINISTRATION 

Capt.  George  Trever 
AFPRO-TRW 


COMPUTER  SOFTWARE 
CONTRACT  ADMINISTRATION  OVERVIEW 


•  APPROACH 

•  FIELD  STUDY 

•  PLANNING  CONSIDERATIONS 


COMPUTER  SOFTWARE 
CONTRACT  ADMINISTRATION  APPROACH 

•  CONDUCT  STUDY 

-  FRESH  LOOK  AT 
MANAGEMENT  TOOLS 

-  FIELD  TEST  IDEAS 

•  INVOLVE  ALL  FIELD  DETACHMENTS 

•  ADDRESS  ALL  COMPUTER  SOFTWARE 


AFCMD  MANAGEMENT  TOOLS 


•  CONTRACTOR'S  MANAGEMENT  SYSTEM 
EVALUATION  PROGRAM  (CMSEP) 

-  AFCMD  REGULATION  178-1 

•  DIRECT  SUPPORT  TO  BUYING  AGENCY 

-  DEFENSE  ACQUISITION 
REGULATION  (DAR)  1-406 

~  MEMORANDUMS  OF  AGREEMENT 
(MOA) 


•  WHAT  IS  CMSEP? 

-  AN  EXISTING  AFCMD  MANAGEMENT 
TOOL 

-  PROVIDES  A  HEALTH  CHECK  ON 
CONTRACTOR'S  MANAGEMENT  SYSTEMS 

-  IMPOSED  AT  ALL  AFPROs 

•  WHAT  DOES  CMSEP  PROVIDE? 

-  ASSURES  DOCUMENTATION  AND  USE  OF 
SOUND  MANAGEMENT  PRACTICES 

-  PROMOTES  PREVENTATIVE  MAINTENANCE 
OF  MANAGEMENT  SYSTEMS 


AFCMD  FIELD  STUDY 


•  CONDUCTED  BY  AIR  FORCE  PLANT 
REPRESENTATIVE  OFFICE  (AFPRO)  AT  TRW, 
REDONDO  BEACH,  CA. 

•  LIMITED  TO  ENGINEERING  AND  QUALITY 
ASSURANCE  FUNCTIONS 

•  ACTIVE  PARTICIPATION  BY  TRW  AND  HQ 
AFCMD 

•  OBJECTIVE: 

-  IMPROVE  CMSE  PROGRAM 

-  DETAIL  DIRECT  SUPPORT  FUNCTIONS 


SOFTWARE  ENVIRONMENT 


TYPE  OF  SOFTWARE 

DELIVERABLE 
TO  THE 
GOVERNMENT 

NONDELIVERABLE 
TO  THE 

GOVERNMENT 

SYSTEM  iCPCII 

X 

N/A 

TEST  (CPCII 

ATE 

X 

N/A 

TEST  SUPPORT 

CAT 

CAI 

X 

X 

FIRMWARE 

HWR  INTENSIVE 

SW  INTENSIVE 

X 

X 

ENGINEERING  SUPPORT 

MODELING 

SIMULATIONS 

X 

X 

COMPUTER  OPERATING  SYSTEMS 

X 

X 

CAO/CAM  SYSTEMS 

N/A 

X 

BUYING  AGENCY  SUPPORT  ACTIVITIES 


TEST  MANAGEMENT  SYSTEM  INDICATORS 
IMSIsj 

MSI  T  DESIGN  MANAGEMENT 

•  SW  REQUIREMENTS 

•  DESIGN  REVIEWS 

•  programming  STANDARDS 

•  internal  documentation 

MSI  2  CONFIGURATION  MANAGEMENT 

•  SELECTION  OF  CPCIi 

•  CM  plans 

•  IDENTIFICATION  OF  SW 

•  BASELINE  MANAGEMENT 

MSI  3  TEST  MANAGEMENT 

•  TEST  PLANS.  PROCEDURES,  •  TEST  MANAGEMENT  AUDITS 

REPORT!  ,  SW  CORRECTIVE  ACTION 

•  TESTING  AND  ANALYSIS 

•  TEST  SOFTWARE  VALIDATION 


•  INTERFACE  CONTROL 

•  SW  LIBRARIES 

•  DOCUMENT  MAINTENANCE/STATUS 

•  CONFIGURATION  REVIEWS 


•  DESIGN  MANAGEMENT  AUDITS 

•  SV>  SPECIFICATIONS 

•  SW  DEVELOPMENT  PLAN 


•  OVERVIEW 

-  AFPROs  PERFORM  CONTRACT  ADMINISTRATION 
lAW  DEFENSE  ACQUISITION  REGULATION 
(DAR  1-406) 

-  AFPROs  SUPPORT  BUYING  AGENCIES  BY  MOA 

•  PROPOSED  TASK  LISTINGS 

-  EXPLICIT  DEFINITION  OF  AFPRO  TASKS 

-  DETAIL  TASKS  TO  APPROPRIATE  LEVEL 

-  USE  TO  DEFINE  PERSONNEL  TYPE  AND  SKILL 
NEEDS 

-  MINIMIZE  OVERLAP  WITH  SPO 


SAMPLE  EN/QA  AFPRO  INVOLVEMENT 


EVALUATION  SYSTEM 


•  MANAGEMENT  SYSTEM  REVIEW 


AwOlT  OOCUMIhTS 
OtTAli  SChCOuIE 
AE  VIEWS 


rWCMCCf  INTERWAl 
AAOCtOUAE  AEVIEM 
-  iNKAEACt 
OEElNitlOM 
AEViCWSTANOAAOS 
AMO  COMVtMEKMS 


WilMlSS  iMff  UMAl 
DEMOMSTAAIiONS 
’■  WiTMESSDRV  HUMS 
-  SCHEDULE  StAfuS 


ASS'S^  SAO 
WOMfOlW'TMSS  ■ 
«EUJT$ 


Af  View 

COMBACIDAS 

aiAO'MISS 


-  SURVEY 

-  COMPREHENSIVE  EVALUATION 

•  SIGNIFICANT  MANAGEMENT  CONCERNS 

-  LIBRARY  CONTROL  (NONDELIVERABLE) 

-  DOCUMENTING  FIRMWARE 

-  TEST  AND  ENGINEERING  SUPPORT 
SOFTWARE 

-  DESIGN  AND  PROGRAMMING 
STANDARDS  FOR  NONDELIVERABLE 
SOFTWARE 

•  CMSE  CRITERIA  REVIEW 

-  AFCMD  DETACHMENTS 

-  COMPANIES 

•  REACTION 

-  TEST  SOFTWARE  NEEDING  ATTENTION 

-  INTEGRATION  OF  HARDWARE/ 
SOFTWARE  EVALUATIONS 

-  DEFINING  THE  CRITICALITY  OF 
SOFTWARE 

-  EVALUATING  FIRMWARE  DEVELOPMENT 


'  »COMriCUW*TlOM  COMtMOl  S040D  — 

— — —  •COMTH*C»0*'I^  COMMuWCAt'OMS  — — — 
'HICM  MIlOM'tV  tASBS 


PLANNING  CONSIDERATIONS 


•  NEAR  TERM  OBJECTIVES 

-  ESTABLISH  COMPUTER  RESOURCE  STAFF  OFFICE 

-  PUBLISH  SOFTWARE  CMSE 

-  DEVELOP  GUIDEBOOKS 

•  SAMPLE  ADEQUACY  CRITERIA 

•  RECOMMENDED  DIRECT  SUPPORT  TASKS 

-  CONDUCT  BASIC  SOFTWARE  QA  TRAINING 

•  FUTURE  PLANNING  OBJECTIVES 

-  CONTRACT  REVIEWS 

-  IMPROVED  MOAt 


-  INTERAGENCY  TRAINING  PROGRAMS 


•  IMPROVE  USAF'S  MANAGEMENT  OF  WEAPON  SYSTEM 
EMBEDDED  CDMPUTERS 


•  GIVEN  RECENT  TRENDS,  ESCALATING  SOFTWARE  SUPPORT  COSTS  WILL 
COMPROMISE  USAF'S  MISSION  READINESS. 

•  LONG  DEVELOPME.NT  TIMES. 

•  OPERATIONAL  PHASE  SOFTWARE  PROBLEMS-STILL  GENERALLY  POOR. 

•  AN  INCREASINGLY  MORE  CAPABLE  AND  TECHNICALLY  SOPHISTICATED 
ENEMY. 


•  A  LIMITED  BUDGET  AND  CONTINUED  BUDGET  REDUCTION 
PRESSURES 

•  NEW  TECHNOLOGY  STILL  EVOLVING 

•  LACK  OF  STANDARDIZATION  IN  DEVELOPMENT  OF  HARDWARE 
AND  SOnWARE 

•  PROLIFERATION  OF  EMBEDDED  COMPUTERS  &  LANGUAGES  IN 
NEW  SYSTEMS 

•  DISAGREEMENT  AS  TO  WHAT  TYPE  OF  SUPPORT  FACILITY 
SHOULD  BE  BUILT  AND  WHO  SHOULD  OWN  IT  IN  THE  C’  AREA 

•  A  HISTORY  OF  CHANGE  OVER  THE  LIFE  CYCLE  OF  A  WEAPON  SYSTEM 


•  COST  VIISUS  COMPIIXIIT 

•  «.l  OVM  NXIOWUI  IT  ITU 

•  CONTOtCTOI'S  tlUSSMINI 


*  DISCIPLINED  SYSTEM  DEVELOPMENT  ENVIRONMENT. 

*  MORE  EMPHASIS  ON  STANDARDIZING  LANGUAGE 
AND  SUPPORT  SYSTEM 

*  MORE  INTERACTION  NEEDED  BETWEEN  DEVELOPER. 
OPERATOR  ANO  MAINTAINER. 
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CORE  CONCEPT 
Major  Jim  Riley,  ALD 


BUT . 

•  PROBLEMS  WITH  6FE  OF  HARDWARE/SOFTWARE? 

•  STIFLE  COMPETITION? 

•  STIFLE  TECHNOLOGY? 

•  LIHLE  ORGANIC  SOFTWARE  SUPPORT  IN  SPACE  PROGRAMS 


SOLUTIONS 


•  GOOD  MANAGEMENT 

•  STANDARDIZE  ONLY  WHERE  IT  MAKES  SENSE 

•  B-52  OAS  EXAMPLE 

•  MATURE  STANDARDS 

•  STEPWISE  INFUSION  OF  NEW  TECHNOLOGY 

•  STANDARDIZE  AT  INTERFACE  LEVEL 

•  APPLICABILITY  TO  SPACE  PROGRAMS 

•  SD  SHOULD  INVESTIGATE 

•  REQUIRE  LARGE  SCALE  CONTRACTOR  INPUT 


POTENTIAL  BENEFITS 


!• 


•  CONTROL  COSTS 

•  POTENTIALLY  SHORTENED  DEVELOPMENT  TIMES 

•  IMPROVED  S/W  RELIABILITY  l,E.  FEWER  ERRORS 

•  IMPROVED  RESPONSE  TO  CHANGE,  I.E.,  EW  CHANGES  (QRC) 

•  EXPEDITE  MANAGEMENT  DECISION  PROCESS 


B 
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SOFTWARE  QUALITY  ASSURANCE 
R.L.  Evans 

Command,  Control,  Comm.  Company 

GUIDEBOOKS  WRITTEN 
FOR  SERVICES  BY 
CONTRACTORS- 

DEFINED  "ACTIONS"  ASD/ESD/SAflSO 
WERE  TO  TAKE  RELATIVE  TO  SWQA 

SOFTWARE  nUALITY  ASSURAilCE  PRESENTATION  TO  THE 
1080  CONFERENCE  ANP  WORKSHOP  ON  MISSION  ASSURANCE 

WHAT  IS  THIS 
THING  CALLEP 

SOFTWARE  QUALITY  ASSURANCE 

PROLIFERATION  OF  MIL-SPEC'S 

MIL-S-52779(AD) 
f'1IL-S-52779A 
AFR  300-1A 
riIL-STD-1679 
MIL-STD-16Ai| 

SCHOOLS  ARE  GIVING  COURSES  IN  SWQA 

1)  I  .'HAT  no  THEY  TEACH? 

2)  HOW  DOES  IT  RELATE  TO  HARDHARE/SYSTEri  QA? 

3)  QA  TOOLS  DEVELOPriENT/IMPLEMENTATION? 

11)  COMPARISON  BETWEEN  HARDWARE/SOFTWARE? 


ARE  COriPANY  stanparhs  realistic  Ann 
BEING  IflPLEflENTED  OR  ARE  THEY  STRICKLY  P.R, 

ARE  RFP'S/CONTRACTS  REALISTIC  RELATIVE  TO 
SOFTWARE  PA  PROVISIONS  -  WHAT  IS  SWOA 
OTHER  THAN  AS  DEFINEO  IN  niL-S-S-5277?,  niL-STP-lC7C  E  -16RA 
AND  WHAT  DOES  IT  flEAN  TO  A  1  HILLION, 
lOfC  2011,  $  CONTRACT. 


MOST  SWQA  ACTIVITIES  HAVE  BEEN 
DISCUSSED  ANP  REPISCUSSED  -  BASED 
ON  THESE  DISCUSSIONS  AND  WORKSHOP'S 
WHAT  HAS  HAPPENED?  ARE  SWQA 
ACTIVITIES  HAPPENING? 

ARE  SWQA  PLANS  BEING  IMPLEMENTED? 
AUDITED?  WflAT  ARE  RESULTS  OF  AUDITS? 
HAVE  CORRECTIVE  ACTION  SYSTEMS  BEEN 
DEVELOPED?  ARE  LESSONS  LEARNED 
TRANSPORTED  TO  NEW  SW  SYSTEM 
DEVELOPMENTS? 


RECOUIENHATIONS  - 

NEED  TO  ESTABLISH  SWOA  WITHIN  THE  OA  ACTIVITIES  -  B^TH 
HARDWARE/SOFTWARE  FOLKS  CAN  LEARN  Fonr:  [ACH  OTHER  -  HEED 
TO  ESTABLISH  A  "FEEnBACK"  SYSTEIl  FOP  SCFTl.'APF.  - 
0  OA  FinO  DEFECTS  EARLY  -  FEEDBACK  OF  THE  NECESSARY  INFORIIATION 
WHICH  WILL  PREVENT  PRODUCTION  SUBSTANDARD  PRODUCT, 

0  QA  COriBIflEO  EFFORT,  HAVE  AUDIT  TE."!!  DOUBLE  CHECK  THE 
RESULTS  FOUND  DURIIIG  INSPECTION, 

0  ESTABLISH  COd'lUNICATIOHS  NETWnRK  -  HAVE  SYSTEIi  '..'MICH 
WILL  GUIDE  OA  PERSONNEL  THROUGH  THE  PPOBLEW  SOLVING  PROCESS. 

0  PUBLISH  FINDINGS  -  THE  RESULTS  OF  SW^A  EFFORTS  OFFER 
ADDITIONAL  GUIP.ANCE  TO  TOP  LEVEL  I’ANAGEOfNT;  CHAPT  FINDINGS; 
PUBLISH. 
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CONCLUSION: 

QUALITY  ASSURANCE  ORGANIZATION 
MUST  TAKE  RESPONSIBILITY 
FOR  ALL  OA  ACTIVITIES  INCLUDING 
SOFTL'ARE. 

OA  fiUST  ESTABLISH  AUDIT  ACTIVITIES 
TO  ASSURE  STANDARDS  ARE  BEING 
FOLLOWED. 

OA  PERSONNEL  HUST  BE  TRAINED  TO 
UNDERSTAND  THE  S'.!  PRODUCTION  PROCESS 
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PRIMARY  FLIGHT  SOFTWARE  SYSTEM 
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ABSTRACT 


The  onboard  Space  Shuttle  flight  software  system,  developed  under  con¬ 
tract  to  NASA,  is  responsible  for  virtually  all  onboard  functions  related  to 
vehicle  guidance,  navigation,  and  flight  control.  In  addition.  It  performs 
numerous  vehicle  systems  management  and  monitoring  functions  as  well 
as  providing  vehicie-to-crew  and  vehicle-to-ground  interfaces.  Unlike  many 
commercial  and  aerospace  software  systems,  the  Shuttle  software  can¬ 
not  be  incrementally  tested  under  operational  conditions  before  delivery  to  the 
customer.  The  software  for  the  Shuttle’s  first  flight  wiil  not  undergo  suborbital 
or  unmanned  test  flights.  Instead,  the  integrated  system  will  be  expected  to 
operate  flawlessly  the  first  time  it  is  placed  in  an  operational  environment. 
Since  the  software  package  is  an  integral  and  critical  part  of  the  Shuttle 
system,  a  development  and  testing  approach  has  been  employed  which 
addresses  the  unique  Shuttle  requirements,  namely,  to  provide  software  which 
meets  the  letter  of  the  contractual  requirements,  performs  in  accordance  with 
Shuttle  operational  requirements,  and  Is  “error-free.”  The  organization 
charged  with  the  development  and  delivery  of  this  complex  system  differs  from 
other  software  development  groups  in  that  it  contains  a  totally  independent 
line-organization  whose  purpose  is  to  provide  final  verification  before  delivery, 
that  is,  prove  the  system  is  “error-free”  and  therefore  ready  for  flight.  The 
verification  organization  operates  in  parallel  to  the  development  groups  but 
maintains  complete  independence.  During  the  development  phase  of  the  soft¬ 
ware  contract  the  Verification  group  gains  familiarization  with  the  detailed 
requirements  and  the  evolving  design,  and  finalizes  a  comprehensive  test  pian 
to  be  approved  by  the  customer.  This  test  plan  is  approached  with  the  attitude 
that  the  software  is  essentially  untested.  Following  completion  of  the  develop¬ 
ment  phase,  the  software  is  placed  under  configuration  control  by  the  Verifica¬ 
tion  group.  It  is  then  subjected  to  an  extensive  series  of  code  inspections, 
detailed  logic  tests  and  overall  system  performance  tests  executed  on  a  simu¬ 
lator  which  utilizes  actual  Shuttle  computer  hardware.  All  discrepancies  are 
documented,  corrected,  and  the  area  of  the  software  which  was  altered  by  the 
correction  is  re-verified.  At  the  completion  of  the  verification  testing,  the 
customer  reviews  the  entire  test  program  and  selected  test  results.  The  cus¬ 
tomer  then  assumes  responsibility  for  configuration  control  of  the  software. 
Shuttle  experience  to  date  has  deviated  somewhat  from  the  strict  serial 
approach  of  development,  verification,  and  delivery.  Experience  has  shown 
that  it  is  advantageous  to  release  software  for  use  in  NASA  trainers  and 
integration  laboratories  at  the  same  time  it  is  delivered  for  verification.  This 
approach  adds  "shelf-life”  to  the  software  without  the  expense  of  additional 
"elapsed-time.”  The  approach  also  fosters  early  identification  of  discrepancies 
and  needed  requirements  changes. 
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INDEPENDENT  VERIFICATION  AND  VALIDATION  TESTING 

OF  THE 

SPACE  SHUTTLE  PRIMARY  FLIGHT  SOFTWARE  SYSTEM 


1.  INTRODUCTION 

The  devalopmvnt  and  verification  of  the  Shuttle  onboard  Pri¬ 
mary  Avionica  Software  Syttam  (PASS)  haa  poaad  unique 
challangaa  asioclatad  with  taw  other  aaroapaca  or  commercial 
software  syatams.  Thaaa  challengea  stem  from  Its  aize  and  com¬ 
plexity,  Its  criticality  to  completion  of  the  Shuttle  mission,  and 
from  the  fact  that  It  la  only  one  of  many  components  of  an 
overwhelmingly  complex  state-of-the-art  Space  Transportation 
System  (STS). 

With  respect  to  size  and  complexity,  the  software  being  rea¬ 
died  for  the  first  orbital  flight  test  (STS-1 )  of  the  Shuttle  Is  actu¬ 
ally  eight  separately  executable  programs  or  memory 
configurations  sharing  a  common  operating  system.  These  pro¬ 
grams  are  stored  on  a  mass  memory  tape  device  and  are  loaded 
into  the  onboard  computers  on  crew  request  (see  Figure  1).  One 
ot  the  eight  performs  all  functions  necessary  to  the  launch, 
ascent,  and  orbit  Insertion  phases  of  the  mission.  Two  others 
provide  software  to  be  used  during  on-orbit  operations  and 
Include  both  navigation  and  control  functions  as  well  as  system 
monitoring  and  management  functions.  Another  memory  con¬ 
figuration  performs  all  functions  necessary  to  the  de-orbIt,  entry, 
and  landing  mission  phases.  The  remaining  four  are  designed  to 
perform  critical  pre-launch  and  on-orbIt  vehicle  checkout  proce- 
durea.  In  all,  these  eight  programs.  Including  tha  software  oper¬ 
ating  system,  comprise  approximately  one-halt  million  32-bit 
words  of  data  and  executable  Instructions.  All  Shuttle  applica¬ 
tions  software  modulea  and  a  large  part  of  tha  operating  system 
Included  In  these  memory  configurations  are  Implemented  In  tha 
HAL/S  high  order  language.  The  language  and  asaociated  com¬ 
piler,  specifically  developed  for  aerospace  applications,  are 
scientifically  oriented  and  especially  suited  to  realtime  syatema. 

The  criticality  of  the  software  to  tha  Shuttle  is  emphasized  by 
Its  numerous  Interfaces  with  other  vehicle  subsystems.  There  are 
taw  functions  Integral  to  the  Shuttle  operation  for  which  the 


software  does  not  perform  some  type  of  computational  service. 
Specifically,  the  onboard  software  la  responsible  for  the  guid¬ 
ance.  navigation,  and  flight  control  functions  performed  during 
all  flight  phases.  This  Includes  both  the  gathering  of  environment 
and  sensor  Input  data  and  the  issuing  of  commands  to  the  vehi¬ 
cle  effectors  (engines,  aeroaurfacea,  ate.).  Tha  software  also 
handles  all  vehicla/ground  interface  functions  pre-launch 
through  landing  via  a  direct  launch  data  bus  (ground)  or  teleme¬ 
try  (In  flight) .  In  addition.  It  provides  many  crew/aubaystem  Inter¬ 
faces.  Other  software  functions  Include  the  management  and 
monitoring  of  onboard  syatams,  fault  detection  and  annuncia¬ 
tion,  and  pre-flight  and  pre-entry  checkout  and  sating 
procedures. 

The  number  and  size  ot  the  services  performed  by  the  onboard 
software  are  not  the  only  factors  contributing  to  Its  complexity. 
The  requirement  for  redundancy  to  achieve  reliability  has  also 
baen  a  factor.  To  obtain  the  required  "Fall-operatlonal/Fall-safe" 
reliability,  the  software  in  certain  critical  flight  phases  must  exe¬ 
cute  redundantly  In  multiple  computers.  For  example,  during  the 
STS-1  mission  the  software  tor  the  Ascent  and  Entry  phases  will 
execute  redundantly  In  lour  of  the  five  IBM  System/4  PI  Model 
AP-101  Qeneral  Purpose  Computers  (OPCs),  the  fifth  computer 
being  reeerved  tor  the  Backup  Flight  System  (BFS)  (see  Figure 
2).  To  achieve  this  redundancy,  an  Intercomputer  synchroniza¬ 
tion  scheme  which  guarantees  Identical  inputs  and  outputs  from 
the  redundant  computers  had  to  be  developed.  The  software 
needed  to  support  this  redundancy  requirement  Is  an  Integral 
part  of  the  operating  system  architecture.  It  provides  such  func¬ 
tions  as  computer  synchronization  at  rates  up  to  330  times  per 
second  and  control  of  Input  data  to  assure  that  all  computers 
receive  Identical  Information  from  redundant  sensors  whether  or 
not  hardware  failures  have  occurred.  A  detailed  discussion  of 
the  Shuttle  redundant  computer  operation  can  bo  found  In 
Reference  1. 
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Figure  1.  Shuttle  Mission  Profile  and  Software  Memory  Configurations 


2.  SOFTWARE  DEVELOPMENT  AND  TEST  ORGANIZATION 

Qotls  »nd  06/acr/ira« 

Tha  organizational  atructura  raaponalbla  for  tha  davalopmant 
and  taat  of  tha  Shuttia  software  evolved  from  previous  space 
software  development  programs  Including  Apollo  and  Skylab. 
The  most  beneficial  experience,  however,  was  gained  during  the 
Approach  and  Landing  Teat  (ALT)  phase  of  the  Shuttia  program 
In  which  a  small  portion  of  the  onboard  software  was  developed, 
tasted,  and  flown.  Although  this  system  was  considerably 
smaller  and  tha  flight  environment  benign  In  comparison  to  the 
STS-1  mission,  valuable  software  design,  test,  and  managerial 
axparlanca  directly  applicable  to  the  STS-1  activity  was  gained. 

The  primary  objectives  of  the  STS- 1  software  organization  can 
be  summarized  In  three  key  points.  The  first  and  most  obvious  Is 
to: 

e  Develop  software  which  adlwrsa  to  the  letter  of  the 

eustemei's  raqulramenla. 

This  refers  to  formal  requirements  documentation.  Secondly, 

o  To  assure  that  the  software  performs  In  accordance 
with  the  Customer's  operational  oapeetatlona  for 
both  nominal  and  off-nomlnal  conditions. 


This  refers  to  operatlonel  requirements  explicitly  or  Implicitly 
Identified  In  crew  procedures,  operational  mission  profiles,  etc. 
And  finally, 

o  To  provide  software  which  Is  "error-free." 

The  first  two  goals  are  easily  quantifiable  since  specific  and 
tasting  plans  can  be  Implemented  to  achieve  them.  For  Instance, 
specific  test  scenarios  can  be  developed  to  cover  all  documented 
software  requirements  and  a  one-to-one  mapping  maintained 
and  reported.  This  has  been  done  tor  the  STS-1  software  and  will 
be  discussed  in  Section  5. 

The  third  goal  of  producing  error-tree  software  Is  a  more 
nebulous  objective  when  applied  to  complex  systems  and,  as 
experience  has  shown,  can  only  be  asymptotically  approached. 
How  well  a  software  development  and  test  organization  does  In 
achieving  this  goal  depends  on  how  effectively  It  Is  structured  to 
address  the  following  areas: 

•  Earl  /  Identification  and  application  of  programming 
standards  and  techniques  with  subsequent  checks 
on  the  fidelity  with  which  these  have  been  followed. 

•  Establishment  of  tests,  audits,  and  code  Inspections 
not  specifically  driven  by  customer  requirements 
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GNC:  Guidmce,  Navigation,  And  Control 


Figure  2.  Space  Shuttle  Data  Proceasing  System 


but  necessary  to  prove  the  design  error-free.  Specif¬ 
ic  examples  of  these  will  be  mentioned  later  during 
the  discussion  of  detalled/functlonsl  verification  In 
Section  5. 

•  Early  definition  of  requirements  for  the  simulation 
teat  bed(s)  and  other  support  software  accompanied 
by  a  thorough  test  plan  and  subsequent  configura¬ 
tion  control. 

e  Configuration  control  of  the  Incremental  build  and 
Integration  of  the  evolving  software  system. 

e  Configuration  control  of  the  Implementation  and 
ratest  of  the  software  changes  resulting  from  re¬ 
quirements  upgrades  and  discrepancy  corrections. 

Functional  Artas 

The  organization  chosen  to  address  each  of  these  key  Items  Is 
divided  Into  five  functional  areas.  Each  represents  a  line  depart¬ 
ment  although  all  are  not  equal  in  size  or  position  within  the 
organizational  hierarchy.  They  are: 

e  Requirements  Analysis  and  System  Architecture 

e  Software  Development 

e  System  Integration  and  Build 


•  Independent  Verification 

•  Customer  Support  and  Field  Teat. 

Software  Daalgn,  Davalopmant,  and  Inlagratlon 

The  first  three  functions,  found  In  most  software  organiza¬ 
tions.  are  responsible  for  the  design,  Implementation,  Integra¬ 
tion,  and  module-through-system-level  test  of  the  software.  The 
Requirements  Analysis  group,  a  "front-end"  organization, 
oriented  toward  applications  software.  Is  made  up  of  engineers 
and  programming  specialists  familiar  with  avionics  systems. 
Their  participation  started  with  Involvement  In  the  early  formula¬ 
tion  of  the  software  requirements  by  NASA  and  tha  prime  con¬ 
tractor.  Rockwell  International.  Their  role  was  to  assess  the 
feasibility  of  Implementing  the  requirements  Into  the  date  pro¬ 
cessing  hardware  and  software.  A  second  objective  was  to  gain 
an  In-depth  understanding  of  the  choeen  avionics  design  to 
guide  their  subsequent  system-level  software  design  activities. 
This  understanding  of  both  the  avionics  design  and  the  software 
structure  also  proved  Invaluable  to  requirements  analysts  In  their 
later  role  as  advisors  to  the  programmers  Implementing  the 
detailed  software  design. 

The  System  Architecture  organization  performs  a  role  similar 
to  the  Requirements  Analysis  group  with  their  efforts  primarily 


directed  toward  the  operating  ayatem  aoftware.  In  addition,  thair 
reaponsibllltlea  Included  the  sizing  and  measurement  of  the 
evolving  software  package  (CPU  utilization,  memory  size,  timing 
measurements). 

Shuttle  software  development  la  divided  Into  two  major  organi¬ 
zations,  one  responsible  for  the  operating  ayatem  and  aaaoclated 
user  interface  software,  and  the  aecond  reaponalbla  for  the 
"applications"  software  (e.g.,  navigation  aoftwara,  flight  control 
software,  etc  ).  These  organizations,  assisted  by  the  require¬ 
ments  analysts  and  system  architecture  personnel,  design  and 
code  the  software  at  the  module  or  subroutine  level.  They  also 
perform  Initial  Integration  by  coupling  Individual  modulea  with 
an  executive  structure  to  produce  overall  subsystems  such  as 
the  Guidance,  Navigation,  and  Flight  Control  Software.  Teating 
Is  performed  at  each  level  of  development  from  the  module  level 
using  driver  programs,  through  the  Initial  Integration  level. 

The  final  Integration  of  applications  programs  with  the  operat¬ 
ing  ayatem  Is  performed  by  the  System  Integration  group.  It  la 
this  group's  responsibility  to  collect  new  or  updated  aoftware 
modules  from  the  development  organizations,  compile  or 
assemble  them,  and  use  the  resulting  object  code  to  update  a 
master  software  ayatem  library.  This  master  system  Is  then  link- 
edited  and  paased  through  a  mass  memory  build  program  which 
converts  It  to  a  form  which  can  be  uaed  to  load  a  Shuttle  onboard 
mass  memory  tape  unit.  The  combined  Integration  and  Develop¬ 
ment  organizations  perform  additional  system  level  tests  after 
the  final  systems  have  been  built.  These  Include  nominal  flight 
simulations  In  a  multicomputer  environment  as  well  as  hard- 
wars/software  Interface  teats. 

As  s  part  of  their  development  and  Integration  raaponalbllltles, 
theaa  three  organizations  define,  apply,  and  enforce  program¬ 
ming  standards  and  techniques.  They  also  maintain  configura¬ 
tion  control  during  all  levels  of  design,  coding.  Integration,  and 
testing.  This  control  pertains  not  only  to  the  software  package 
but  also  associated  documentation  such  as  requirements  base¬ 
lines,  design  descriptions,  discrepancy  tracking,  and  Integration 
and  system  build  documentation. 


Ind»p»ndtnt  V»rHlc»Uon 

Independent  Verification,  which  will  be  discussed  at  length  In 
Section  5,  Is  a  separate  line  organization  with  the  ultimate 
responsibility  of  assuring  that  the  software  Is  error-free.  The 
Verification  group  maintains  an  equal  status  with  the  Software 
Development  organization,  and  derives  Its  authority  by  remain¬ 
ing  completely  Independent  with  respect  to  management  and 
personnel.  The  Verification  organization  for  the  Shuttle  PASS  Is 
shown  In  Figure  3. 

The  philosophy  of  the  Verification  group  la  based  on  the  pre¬ 
mise  that  the  software  Is  untested  when  received.  This  is  a  key 
factor  In  the  development  of  their  test  plan  Because  their  role  Is 
essentially  that  of  a  pseudo-customer,  they  maintain  an  adver¬ 
sary  relationship  with  the  Development  groups.  Verification 
receives  the  aoftware  at  the  end  of  the  design,  development,  and 
integration  cycle  and  subsequently  subjects  It  to  a  complete 
verlfication/valldatlon  program.  The  Ideal  timeline  for  the  devel¬ 
opment,  verification,  and  delivery  of  the  aoftware  will  be  dis¬ 
cussed  In  Section  3. 

CustoiriBr  Support  artd  Flpld  Tb$i 

The  Customer  Support  and  Field  Test  activity  Ideally  begins 
after  the  formal  Verification  testing  is  complete  and  the  system 
has  been  Inspected  and  accepted  by  the  customer.  In  reality,  this 
activity  occurs  In  parallel  with  Verification  tor  reasons  which  will 
be  discussed  later.  This  organization  provides  a  number  of  ser¬ 
vices  to  software  users  at  customer  test  sites  (simulators)  and  on 
the  actual  Shuttle  vehicle.  These  Include:  Installation  and  check¬ 
out  of  the  newly  released  software;  problem  analysis,  trouble¬ 
shooting  and  Implementation  of  temporary  solutions;  customar 
education  and  customer/software  organization  liaison,  test  site 
unique  software  alterations;  customer  test  program  support; 
crew  training  program  support;  mission  support. 

3.  SOFTWARE  DEVELOPMENT  LIFE  CYCLE 

Figure  4  describes  a  typical  software  development  and  test 
cycle.  The  chart  presents  a  simplified  and  somewhat  Idealized 
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Figure  4.  Software  Development  Life  Cycle 


view  of  the  cycle,  deecribing  cuetomer  and  organizational  inter¬ 
faces  as  well  as  the  flow  of  design,  development,  test,  delivery, 
and  maintenance  In  general,  STS-1  experience  has  shown  the 
flow  to  be  valid  Variations  have  been  associated  either  with  the 
phasing  of  activities,  or  with  the  number  of  times  each  was 
performed  The  relationship  between  the  current  STS-1  software 
delivery  schedule  and  this  ideal  flow  will  be  discussed  In  Section 
6.  entitled  "Field  Test  -  Early  Customer  Involvement." 

Figure  4  shows  the  three  phases  of  the  requirements  definition 
process.  System-level  requirements  address  the  overall  system 
architecture,  including  the  operating  system  and  user  interface 
software.  Functional  level  requirements  describe  high-level 
applications  software  interfaces,  general  capabilities,  and  timing 
requirements.  Examples  Include  guidance,  navigation,  and  flight 
control  software  functional  requirements,  and  the  vehicle  check¬ 
out  software  functional  requirements.  Detailed  requirements 
define  equations,  algorithms,  initialization  data,  and  formats  for 
all  areas  described  in  the  functional  requirements.  Examples  are 
redundancy  management,  ascent  navigation,  telemetry  uplink, 
etc.  Definition  of  these  phased  software  requirements  was  not 
unilateral  but  was  performed  by  design  teams  with  representa¬ 
tion  from  all  areas  of  the  Shuttle  engineering  community,  both 
NASA  and  contractors.  IBM  participation  Included  representa¬ 
tives  from  both  the  software  design  and  development  organiza¬ 
tion  and  the  Verification  group.  Final  baselining  of  the 
requirements  documents  was  the  responsibility  of  the  NASA 
Space  Shuttle  Orbiter  Project  Office. 

Both  the  software  development  and  verification  activities  are 
Initiated  early  In  the  requirements  definition  period.  The  devel¬ 
opment  activity  progresses  through  a  high-level  functional 
design  to  a  detailed  design  and  teat  planning  phase.  Contractual 
documentation  for  each  phase  is  again  baaellnad  by  the  NASA 
Space  Shuttle  Orbiter  Project  Office  before  proceeding  to  the 
coding  and  Integration  of  the  software.  The  mainline  develop¬ 
ment  activity  culminates  with  the  First  Aitlcls  Configuration 
Inspection  (FACI)  where  the  Integrated  software  package  and  all 
supporting  documentation  Is  dallvered  to  the  Verification  organ¬ 
ization.  At  this  point  the  software  may  also  be  made  available  to 
the  customer  for  use  in  simulator  development,  crew  training, 
and  early  vehicle  bulld-up  activities. 

Prior  to  the  FACI,  responsibility  for  configuration  control  of 
the  evolving  softwara  rests  with  the  Software  Development  and 
Integration  organization.  Following  the  FACI,  this  responsibility 
is  assumed  by  the  Verification  group.  Also  at  this  point  Develop¬ 
ment  and  Integration  assume  a  support  role.  Implementing 
requirements  changes,  correcting  software  discrepancies,  and 
providing  updated  software  relsaaes. 

As  mentioned.  Verification  activities  start  early  in  the  require¬ 
ments  definition  phase.  A  test  plan  is  first  developed  defining  the 
organization,  schedule  tamplatas  and  deliverables.  This  Is  fol¬ 
lowed  by  development  of  Verification  tost  specifications,  which 
are  high-level  descriptions  of  the  proposed  testing  based  on  the 
software  requirements.  These  are  reviewed  and  approved  by 
NASA.  Test  procedures  which  consider  both  the  requirements 
and  the  software  design  sre  then  developed.  These  are  detailed 
test  descriptions  which,  in  turn,  can  be  used  to  build  flight  simu¬ 
lations  and  test  scenarios.  Ideally,  all  Verification  teat  cases  and 
simulations  are  executed  and  thoroughly  checked  out  before  the 
FACI  using  the  early  software  release.  Formal  Verification  then 


commences  with  additional  planned  releases  of  the  software  to 
incorporate  changes.  Verification  ends  at  the  Configuration 
Inspection  (Cl)  with  a  formal  delivery  of  the  software  to  NASA 
and  a  review  of  Verification  testing  by  the  Space  Shuttle  Orbiter 
Project  Office. 

Following  the  Cl,  the  software,  now  under  NASA  configuration 
control,  is  delivered  for  integration  with  other  avionics  subsys¬ 
tems.  Subsequent  avionics  Integrated  tests  In  both  simulators 
and  on  the  actual  Shuttle  vehicle  are  then  performed  by  NASA- 
/Rockwell  with  the  support  of  an  IBM  software  maintenance 
organization.  This  testing  culminates  with  a  NASA  Flight  Readi¬ 
ness  Review  (FRR)  at  which  all  Integrated  Shuttle  systems 
Including  the  software  are  accepted  for  flight. 


4.  DESCRIPTION  OF  TEST  FACILITIES 

Shuttle  program  test  requirements  necessitate  use  of  three 
facilities  for  verification  of  the  Shuttle  avionics  system:  the  Soft¬ 
ware  Development  Laboratory  (SDL),  the  Shuttle  Avionics  Inte¬ 
gration  Laboratory  (SAIL),  and  the  Flight  Systems  Laboratory 
(FSL).  A  fourth  facility,  the  Shuttle  Mission  Simulator  (SMS), 
although  not  apeclfically  Involved  in  testing  the  avionics  system, 
does  play  a  major  role  In  exercising  the  integrated  avionics 
design  through  its  extensive  crew  training  activitisa. 

The  Software  Development  Laboratory  (SDL),  located  in 
Houston  at  the  Johnson  Space  Center,  Is  the  primary  facility  for 
the  development,  integration,  and  verification  of  the  primary 
onboard  software  system.  It  was  developed  and  is  maintained  by 
the  software  contractor  (IBM)  under  contract  to  NASA.  Its  capa¬ 
bilities  will  be  discussed  later  In  this  section. 

The  Shuttle  Avionics  Integration  Laboratory  (SAIL),  also 
located  at  the  Johnson  Space  Center  In  Houston,  Is  responsible 
for  avionics  system  Integration  and  hardware/software  certifica¬ 
tion.  It  Is  primarily  used  by  NASA  and  the  Shuttle  Integration 
contractor  (Rockwell)  to  support  elements  of  their  avionics  veri¬ 
fication  plan.  The  SAIL  was  designed  as  a  simulation  laboratory 
where  avionics  hardware  (or  simulations  of  the  hardware),  flight 
software,  flight  procedures,  and  ground  systems  will  be  fully 
Integrated  and  tested.  The  SAIL  also  provides  realtime  mission 
support  and  post-flight  evaluation  capabilities.  Its  Integration 
and  test  role  tor  STS-1  emphasizes  the  pre-launch,  ascent, 
aborts  and  orbit  insertion  phases  of  the  Shuttle  mission. 

The  Flight  Systems  Laboratory  (FSL),  located  at  the  Rockwell 
facility  In  Downey,  California,  has  capabilities  similar  to  those  of 
the  SAIL.  Its  role  in  the  Shuttle  program  Is  siso  similar  in  that  It  Is 
used  by  NASA  and  Rockwell  to  perform  avionics  system  Integrs- 
tion  and  verification  for  the  on-orbIt,  de-orbit,  entry  and  landing 
phases  of  the  mission.  Both  the  FSL  and  SAIL  are  full  six-degree- 
of-freedom  flight  simulators  with  man-in-the-loop  capability. 

The  Shuttle  Mission  Simulator  (SMS)  is  NASA's  primary  train¬ 
ing  facility  for  Shuttle  crews.  Located  at  the  Johnson  Spsce 
Center  In  Houston,  It  provides  a  realistic  environment  lor  training 
crews  in  all  mission  phases  from  pre-launch  through  landing  and 
rollout.  For  STS-1  training,  the  SMS  consists  of  two  simulators, 
both  employing  actual  Shuttle  data  processing  system  hard¬ 
ware.  One.  the  "moving-base"  simulstor,  provides  cockpit 
motion  consistent  with  vehicle  dynamics.  The  second  is  a  "fixed- 
base”  simulator.  Both  provide  extensive  visusi  displays  in  addl- 
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tion  to  a  direct  link  to  the  Mission  Control  Canter,  also  In 
Houston  Although  not  directly  Involved  In  software  testing,  the 
SMS  does  provide  an  Invaluable  benefit  In  this  area.  Its  extensive 
program  of  crew  training  under  off-nomlnel  conditions  tends  to 
subject  the  software  to  a  wide  variety  of  stress  situations.  This 
provides  added  confidence  In  the  performance  and  capabilities 
of  the  system. 


The  SDL  Facility  ■  Tha  Sottwara  fast  Bad 

The  SOL  serves  a  dual  role  In  the  Shuttle  program.  Tha  Primary 
Avionics  Software  System  Is  both  developed  and  vsrifled  In  the 
SDL.  Because  of  Its  dual  role,  the  laboratory  has  been  designed 
with  capabilities  to  analyze,  verify,  maintain,  and  control  Shuttle 
flight  software.  Additionally,  tha  SOL  Is  used  to  Integrate  all 
software  elements  into  an  orbiter  mass  memory  load  tape  used  to 
load  the  Shuttle  mass  memory  tape  unit. 

The  SDL  provides  six  msjor  functions  with  which  to  perform 
development,  verification,  and  Integration  activities  These  func¬ 
tions  sre;  a  program  management  facility;  a  mass  memory  build 
facility:  a  simulator;  a  post-processor;  a  documentation,  analy¬ 
sis.  snd  ststlstics  system;  and  an  array  of  pre-processors. 

The  Program  Management  Facility  (PMF)  provides  the  pro¬ 
grams  and  control  needed  to  build,  maintain,  and  track  program 
data  and  create  deliverable  systems.  The  mass  memory  build 
facility  provides  the  utilities  by  which  Shuttle  software  elements 
are  mapped  Into  the  format  required  by  the  orbiter'a  mass 
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memory  units.  The  mass  msmory  build  also  creates  universal 
patch  format  (UPF)  tapes  used  by  all  Shuttle  teat  sites  to  update 
onboard  mass  msmory  units 

The  pre-processors  provided  by  the  SDL  convert  dsta  required 
for  flight  Into  form  that  can  be  used  by  the  flight  software.  This 
data  Includes  mission-dependent  Initialization  data  (l-LOADs). 
display  formats,  systems  management  measurement  tolerances, 
and  downllst  telemetry  data. 

The  documentation,  analysis,  and  statistics  system  provides 
users  with  the  data  nscessary  to  develop,  analyze,  and  maintain 
the  evolving  software  system. 

The  SDL  simulator  and  post-processor  functions  provide  the 
primary  facility  used  In  the  development  and  vsrification  of  the 
software  system.  Three  simulation  modes  are  available.  The 
Interpretative  Computer  Simulation  mode  (ICS)  provides  an 
emulation  of  the  AP-101  flight  computer  and  Input/output  pro¬ 
cessor  execution  of  code.  The  Functional  Simulation  Mode 
(FSIM)  allows  execution  of  flight  code  using  a  functional  repre¬ 
sentation  of  the  operating  systam.  Tha  third  (and  by  far  the  most 
extensively  used)  mode  Is  the  Flight  Computer  Mode  (FC)  which 
allows  execution  of  the  flight  software  In  actual  Shuttle  data 
processing  system  hardware. 

The  FC  mode  of  the  SDL  simulator  la  tha  only  mods  used 
during  the  Verification  testing  of  the  software.  It  provides  a 
realistic  closed-loop,  slx-degree-of-freedom  simulation  of  the 
Shuttle  operational  environment  and  avionics  hardwars  (see 
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Figure  5).  Available  Shuttle  hardware  elementa  Include  the  AP- 
101  flight  computers  and  aaaoclated  Input/output  proceasors, 
the  crew  Display  Electronics  Units  (DEUs)  and  CRTs,  and  the 
onboard  mass  memory  tape  units.  The  user  has  the  option  to 
select  software  simulations  of  the  DEU  and  mass  memory  hard¬ 
ware.  All  flight  hardware  Is  Interfaced  via  a  spaclal-purpoae 
Flight  Equipment  Interface  Device  (FEID).  This  davlca  not  only 
provides  the  hardware-to-hardware  and  hardware-to-hoat  com¬ 
puter  Interface,  but  also  the  moans  by  which  the  simulation  la 
monitored  and  controlled. 

The  capabilities  provided  by  the  FEID  allow  the  user  to  monitor 
and  control  the  aoftware  execution  at  the  flight  soft  ware  Instruc¬ 
tion  or  data  location  level.  Uaars  have  the  cabablllty  to  “stop"  a 
multi-flight  computer  simulation  on  time,  event,  or  upon  execu¬ 
tion  of  a  particular  Instruction  or  data  location  reference  within 
the  flight  software.  While  the  simulation  Is  stopped,  capabilities 
are  available  via  direct  memory  access  to  alter  or  collect  data 
from  the  flight  computer  main  memory.  Since  the  simulator  la 
also  stopped  during  this  period,  faults  may  also  be  Introduced 
external  to  the  flight  computer  In  the  almuleted  hardware  or 
vehicle  environment.  Upon  completion  of  the  faulting  or  data 
collection  activity,  the  simulation  can  be  restarted  without  loss  of 
continuity  in  Instruction  execution  or  I/O.  This  ability  to  stop  on 
instruction  and  access  or  alter  the  contents  of  the  flight  compu¬ 
ter  without  affecting  the  simulation  Is  a  unique  feature  of  the 
SDL  and  Is  not  available  In  the  SAIL,  FSL,  or  SMS. 

The  major  outputs  of  the  simulator  are  log  tapes  which  contain 
the  flight  software  commands  Issued  and  the  data  received  or 
transmitted.  The  log  tapes  also  contain  diagnostic  data  such  as 
data  snaps,  dumps,  or  traces  requested  by  the  user.  The  SDL 
post-processor  provides  a  data  reduction  and  manipulation 
capability  for  the  log  tapes.  Post-processor  capabilities  allow 
users  to  specify  the  subset  of  logged  data  to  be  processed.  Its 
format,  special  computations  that  are  required,  and  whether  a 
comparison  should  be  made  with  other  simulations. 

Because  of  Its  singular  role  as  the  test  bed  for  the  Shuttle 
primary  onboard  software,  validation  of  the  SDL  has  always  been 
a  prime  concern.  Requirements  for  the  facility  were  defined  very 
early  In  the  software  program  by  future  users  from  ell  IBM  soft¬ 
ware  organizations.  These  requirements  were  besellned.  Imple¬ 
mented,  thoroughly  tested,  end  pieced  under  configuretion 
control  by  NASA.  New  cepablllties  require  NASA  epproval  and 
are  Implemented  and  tested  at  the  detelled  and  system  levels  by  a 
dedicated  development  and  maintenance  group.  Final  validation 
comes  with  use  of  the  facility  by  the  verlous  softwere  test  organi¬ 
zations.  In  particular,  the  Verification  group  subjects  the  SOL  to 
a  thorough  validation  prior  to  the  FACI  as  part  of  their  test  case 
development  and  checkout  activities  (see  Section  5).  SOL  ver¬ 
sions  selected  for  use  In  formal  flight  software  testing  are  placed 
under  strict  configuration  control  by  both  NASA  and  the  user 
organization.  In  order  to  minimize  variations  in  the  test  bed.  only 
those  changes  considered  mandatory  to  the  completion  of  the 
testing  are  Incorporated. 

5.  INDEPENDENT  VERIFICATION  TEETINQ 

The  Software  Verification  group  Is  an  Independent  line  organ¬ 
ization  with  a  test  plan  based  on  the  premise  that  the  soft¬ 
ware  is  totally  untested  when  they  receive  It.  Since  it  Is  In  line 
to  the  delivery  and  acceptance  of  the  software  by  the  customer. 


the  Verification  group  has  the  ultimata  responsibility  of  proving 
that  the  system  (1)  meets  the  letter  of  the  requirements.  (2)  will, 
as  a  system,  perform  in  accordance  with  operational  expecta- 
tlona.  and  (3)  Is  error-free.  As  previously  discussed,  these  are 
overall  organizational  goals  but  Verification  must  demonstrate 
them  to  the  cuatomer  at  the  Configuration  Inspection. 

Types  of  TnUng 

Verification  testing  Is  divided  Into  two  major  categories,  detall- 
/functlonal  testing  and  performance  or  system-level  tasting.  The 
datall/functlonal  tests  address  the  question  of  whether  the  soft¬ 
ware  meets  the  letter  of  the  requirements  and  Is  basically  divided 
by  software  functional  area,  e.g.,  ascent  guidance  programs, 
entry  navigation,  etc.  Performance  or  system  testing  Is  directed 
at  how  the  system  as  a  whole  will  perform  In  both  nominal  and 
off-nominal  or  stress  situations.  As  a  result,  the  performance 
verification  Is  divided  by  mission  phase,  I.e.,  ascent/aborts,  on- 
orbit,  entry.  The  majority  of  Verification  testing  Is  performed  In 
the  Software  Development  Laboratory  (SDL)  using  actual  Shut¬ 
tle  data  processing  system  hardware.  A  small  amount  of  Verifica¬ 
tion  testing  Is  performed  In  other  facllltlee  such  as  the  Shuttle 
Mission  Simulator  (SMS)  and  Shuttle  Avionics  Integration 
Laboratory  (SAIL)  In  order  to  take  advantage  of  their  extensive 
flight  hardware  and  crew  Interfaces.  A  prime  objective  In  both 
types  of  Verification  Is  to  perform  all  testing  on  the  total  unal¬ 
tered  (or  "unscarred")  software  system  rather  than  a  modified  or 
partial  system. 

Figure  6  describes  the  Verification  organization  structure 
which  is  made  up  of  three  departments.  One  Is  responsible  for 
the  detall/functlonal  testing  of  the  operating  system  and  the 
pre-flight  and  on-orbit  checkout  software.  A  second  carries  out 
the  detall/functlonal  testing  of  the  guidance,  navigation,  and 
flight  control  functions  for  all  mission  phases.  The  third  depart¬ 
ment  Is  the  performance  test  group. 

Verification's  organizational  Independence  and  Its  assump¬ 
tion  that  the  software  Is  untested  when  received  have  both  been 
emphasized  earlier.  In  addition,  two  other  attitudes  play  a  key 
role  In  the  Verification  group's  approach  to  testing.  Since  verifi¬ 
ers  are  considered  systems  analysts  and  not  just  'Testers",  they 
are  responsible  for  ettempting  to  diagnose  problems  and  pro¬ 
pose  solutions  when  possible.  This  expedites  Software  Develop¬ 
ment's  analysis  and  correction  of  alleged  software 
discrepancies.  This  responsibility,  however.  Is  tempered  by  the 
overriding  directive  thet .  .  .  "when  In  doubt— assume  the  aoft- 
were  to  be  at  fault.”  This  Is  key  to  providing  error-free  software.  A 
second  attitude  addresses  the  software  requirements  and  can  be 
simply  stated  as .  .  .  'The  requirements  are  not  to  be  considered 
Infallible."  These  attitudes  end  assumptions  cast  Verification  In 
the  role  of  the  conscience  of  the  project  and  foster  a  definite  but 
healthy  adversary  relationship  between  the  Verification  group 
and  the  Software  Development  organizations. 

D»Wtl Functional  Tatting 

The  "detail"  part  of  detall/functlonal  testing  refers  to  the  ex¬ 
plicit  verification  of  all  documented  customer  requirements.  This 
Includes  logic  paths,  algorithms,  decision  blocks.  Initialization 
constants,  etc.  To  achieve  this,  test  cases  are  defined  to  cover 
eech  paragraph  or  explicit  Item  In  the  requirements  documents. 
A  data  base  mapping  of  test  case  to  requirement  peragraph  Is 
maintained  and  delivered  to  the  customer. 
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Figure  6.  Verification  Departmental  Organization 


Functional  testing  refers  to  the  expiicit  verification  of  ali  soft¬ 
ware  internal  interfaces,  external  hardware  interfaces,  and  soft¬ 
ware  module  and  events  sequencing.  An  example  of  the 
difference  between  detail  and  functional  testing  is  shown  in 
Figure  7.  The  figure  depicts  the  software  system  as  viewed  by  an 
analyst  assigned  the  tasK  of  verifying  the  navigation  software  for 
a  particular  flight  phase.  Referring  to  the  figure,  the  analyst 
would  perform  detailed  verification  of  the  internal  paths  by 
extracting  and  analyzing  data  from  an  SOL  simulation  at  the 
entrance  and  exit  nf  each  path.  Pass/fail  or  evaluation  criteria 
used  in  the  analysis  is  obtained  from  offline  lest  programs,  hand 
computations,  or  direct  comparison  to  requirements.  This  pro¬ 
vides  assurance  that  the  software  adheres  to  the  detailed 
requirements. 

To  satisfy  the  functional  testing  requirement,  the  navigation 
analyst  also  extracts  data  at  points  whore  the  navigstion  soft¬ 
ware  has  input  or  output  interfaces.  Further,  data  is  also  col¬ 
lected  external  to  the  computer  Sensor  input  data  which  can 
affect  navigation  computations  and  output  data  such  as  that 
presented  on  the  crew  displays  Is  also  examined.  This  external 
data  is  correlated  to  that  extracted  at  Internal  points  in  the  soft¬ 
ware.  It  is  obvious  that  this  approach  produces  considerable 
overlap  of  testing  when  applied  by  all  analysts.  However,  this 
overlap  is  considered  an  advantage  when  the  objective  is  error- 
free  software. 

The  SDL  test  cases  used  in  the  above  detail/functional  anal¬ 
ysis  involve  the  execution  of  the  entire  unaltered  software  sys¬ 
tem  The  technique  used  is  similar  to  that  employed  in  flight 
simulators  to  avoid  having  to  initialize  each  simulation  at  a 
dynamical'y  quiescent  point  in  the  trajectory,  e.g.,  at  liftoff. 

In  the  SDL  a  full  ascent,  orbit  or  entry  simulation  is  executed 
and  "reset"  or  "restart"  points  are  selected  at  various  times  In  the 


trajectory.  At  these  points  all  data  necessary  to  initialize  the  flight 
hardware  and  the  host  simulator  is  collected  on  a  mass  storage 
device.  Detall/functlonal  verifiers  then  construct  short  duration 
simulations,  using  these  "reset"  points  to  initialize  their  test  In 
the  region  of  interest.  The  required  analysis  data  Is  then 
extracted  via  extensive  SDL  data  collection  facilities. 

Execution  of  test  cases  Is  only  one  aspect  of  the  responsibili¬ 
ties  of  the  detall/functlonal  analyst.  Associated  with  each  test 
case  Is  e  standardized  checklist  which  must  be  tilled  out  before 
the  test  case  is  considered  completed.  One  item  on  this  checklist 
Is  the  code  Inspection  of  all  software  modules  covered  by  the 
case.  This  "mental  walk"  through  the  software  has  proven  to  be 
especially  fruitful  in  uncovering  particularly  subtle 
discrepancies. 

Early  Approach  and  Landing  Test  (ALT)  software  verification 
showed  that  detail  testing  against  requirements  and  overlapping 
functional  testing  did  not  fully  verify  all  aspects  of  the  software. 
Another  class  of  tests,  called  "common  function  tests",  was 
required.  In  general,  these  are  code  surveys  and  audits,  some  of 
which  are  supported  by  data  collected  from  simulations.  Exam¬ 
ples  include: 

•  a  review  of  the  software  to  determine  whether  all 
divide  operations  are  overtly  protected  against 
overflow  conditions. 

•  verification  that  all  data  is  initialized  at  memory 
reconfiguration. 

•  an  audit  of  all  telemetry  data  to  be  provided  to  the 
software  (UPLINK),  or  to  the  ground  by  the  software 
(DOWNLIST), 

•  a  software  review  to  verity  that  programming  stan¬ 
dards  identified  during  the  software  design  phase 
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Figure  7.  Example  of  Detail/Functional  Verification 


have  been  followed.  This  refers  to  the  use  of  both  the 
HAL/S  language  and  the  AP-101  assembler,  ahd 
standards  imposed  to  ensure  Intercomputer  syn¬ 
chronization 

The  programming  standards  review  Is  probably  the  most 
extensive  survey  performed  on  the  software.  Involving  16  sepa¬ 
rate  surveys  which  must  be  performed  for  the  eight  memory 
configurations 

The  size  of  the  overall  detail/functlonal  verification  effort 
involves  |ust  under  1000  separate  test  cases  and  code  surveys. 
The  STS-1  activity  was  initiated  nearly  three  years  ago  and  has 
involved  approximately  two-thirds  of  the  total  Verification  work 
force 

Pertormance  Tasting  -  Systam-Laval  Tasts 

The  purpose  of  performance  testing  is  to  exercise  the  total 
software  system  in  a  realistic  operational  environment  and  sub¬ 
ject  It  to  off-nominal  stress  situations  The  off-nominal  condi¬ 
tions  introduced  in  the  performance  tests  are  of  two  types.  The 
first  are  "reasonable"  or  expected  failures  and  anomalies  result¬ 
ing  from  hardware  malfunction,  environment  aberations.  or  user 
errors  These  are  anomalies  which  the  Shuttle  has  been 
designed  to  withstand  The  second  type  of  failure  involves  Intro¬ 
duction  of  extreme  or  catastrophic  conditions  including  low 
probability  failures,  and  multiple  or  simultaneous  failures  which 
stress  the  software  and/or  the  Shuttle  system,  at  times  to  the 
breaking-point 


(Rockwell)  to  verify  the  Integrated  avionics  system.  They  involve 
flight  simulations  of  the  various  mission  phases  using  actual 
Shuttle  data  processing  system  hardware.  The  software  cases, 
numbering  about  100  separate  simulations,  are  for  the  most  part 
executed  In  the  Software  Development  Laboratory.  A  small 
number  are  executed  In  the  Shuttle  Mission  Simulator  to  take 
advantage  of  Its  crewman-in-the-loop  capability.  All  simula¬ 
tions  are  developed  using  published  nominal  and  malfunction 
crew  procedures. 

Since  the  performance  cases  in  their  entirety  were  selected  to 
be  a  part  of  the  Cl  Acceptance  Tests  (see  subsection  entitled 
"Test  Documentation  and  Review"),  they  have  received  exten¬ 
sive  attention  from  the  NASA  Project  Office  and  the  general 
avionics  community.  The  content  and  timing  of  the  failures  intro¬ 
duced  in  each  of  the  100  cases,  as  well  as  the  basic  timelines  and 
data  collected  for  analysis,  have  been  the  subject  of  review  for 
the  past  two  and  one-half  years.  After  being  baselined  in  mid- 
1978.  they  were  placed  under  NASA  configuration  control.  In 
addition,  periodic  coordination  meetings  were  held  to  encour¬ 
age  continued  community  Involvement  during  the  development 
and  early  execution  of  the  cases.  These  T est  Coordination  T earn 
meetings,  chaired  by  NASA.  Involved  representatives  from  all 
interested  NASA  and  contractor  organizations.  The  objective  of 
the  meetings  was  to  promote  continued  review  of  test  proce¬ 
dures.  initial  case  results  and  analysis  techniques  in  light  of 
changing  software  requirements  and  updated  mission  timelines. 


The  test  scenarios  for  the  software  performance  cases  are  very  The  performance  cases  themselves  fall  into  two  general  cate- 

similar  to  those  used  by  the  Shuttle  integration  contractor  gories:  guidance,  navigation,  and  flight  control  (GN&C)  cases. 
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and,  system  services  or  operating  system  cases.  The  first,  the 
QN&C  performance  cases,  make  up  about  80%  of  the  testa. 
These  Include  nominal  simulations  for  all  flight  phases  but  are 
for  the  most  part  failure  oriented.  Also  Included  are  "end-to-end" 
(pre-launch  through  landing)  simulations  for  the  nominal  mis¬ 
sion  and  aborts.  Anomalies  Introduced  In  the  GN&C  cases  are  of 
three  types: 

e  failures  or  anomalies  originating  external  to  the  data 
processing  systems,  that  Is,  within  other  avionics 
hardware  or  vehicle  hardware.  Examples  Include 
NAVAID  failures,  engine  failures,  hydraulic  failures, 
etc. 

e  environmental  stress.  Including  vehicle  state 
offsets,  winds,  and  atmospheric  variations. 

e  crew-induced  stress  such  as  early  or  late  moding  of 
the  software  and  delayed  switch  settings. 

The  second  type  of  performance  case,  the  system  services 
stress  test.  Introduces  severe  single  or  multiple  data  processing 
system  or  vehicle  system  failures.  The  failures  are  designed  to 
stress  the  operating  system  software  In  several  critical  areas. 
Including  Input/output  data  processing,  multi-computer  redun¬ 
dancy  management  (I.e..  computer  synchronization),  and  CPU 
utilization  performance. 

The  evaluation  criteria  used  In  the  analysis  of  performance 
cases  generally  consist  of  two  major  activities.  The  first  Is  a 
performance  analysis  of  the  vehicle/avlonics  system.  This  Is 
refered  to  as  a  GN&C  analysis  and  Includes  several  components. 
One  Is  an  engineering  analysis  of  the  actual  response  of  the 
system  vs.  the  expected  response.  A  second  compares  the  vehi¬ 
cle's  performance  against  documented  Infllght/mlsslon  cor 
straints.  These  Include  reference  flight  profiles  and  other 
pass/fall  criteria  used  by  the  avionics  Integration  contractor  In 
their  .certification  program.  A  third  type  of  GN&C  analysis 
Involves  the  plot-on-plot  comparison  of  case  results  against 
Identical  cases  executed  In  other  simulations  (called  "Compari¬ 
son  Tests ")  This  analysis  not  only  serves  to  Identify  problems  in 
the  software  design  but  has  also  been  key  in  providing  confi¬ 
dence  In  the  SOL  simulator  A  final  GN&C  analysis  Is  used  In  the 
evaluation  of  failure  cases  and  Involves  a  plot-on-plot  compari¬ 
son  of  the  case  against  an  Identical  simulation  with  the  failure 
removed 

In  addition  to  a  GN&C  analysis,  all  performance  cases  are 
analyzed  for  software  operating  system  performance.  Special 
purpose  diagnostic  data  collection  performed  on  each  case 
determines  whether  the  operating  system  has  detected  any  soft¬ 
ware  errors  during  the  simulation.  General  system  measure¬ 
ments  are  also  collected  on  most  performance  cases.  These 
include  measurement  of  CPU  utilization  and  timing  constraints 
such  as  transport  lags.  These  measurements  are  used  to  demon¬ 
strate  adherence  to  numerous  system  software  timing  require¬ 
ments. 

Software  Verification  ActMtIea  Prior  to  FACI 

The  software  development  life  cycle  (Figure  4)  showed  Ver¬ 
ification  beginning  during  the  requirements  definition  phase, 
long  before  software  was  available  to  be  verified.  This  lead-time 
Is  necessary  to  build  an  organization  with  the  expertise  to  verify 
the  software  Independent  of  the  design  and  development 
groups,  and  to  develop  a  meaningful  test  plan. 


The  first  step  In  this  process  Involves  development  of  test 
specifications  which  are  high-level  descriptions  of  the  testing  to 
be  performed.  These  are  almost  entirely  based  on  the  software 
requirements  and  contractually  must  be  reviewed  and  accepted 
by  NASA.  This  activity,  along  with  requirements  design  team 
participation,  provides  verifiers  with  an  In-depth  knowlegeof  the 
requirements. 

The  follow-on  test  procedures,  completed  during  the  software 
code  and  test  period,  describe  the  testing  In  detail.  Since  they 
Include  the  test  timeline,  the  data  to  be  collected  for  analysis,  and 
analysis  techniques,  these  test  procedures  must  consider  both 
the  software  design  and  the  requirements.  It  Is  procedures  devel¬ 
opment  that  gives  the  verifier  an  understanding  of  the  Imple¬ 
mented  code. 

Prior  to  formal  delivery  of  the  software  at  FACI.  Verification 
begins  a  test  case  development  and  validation  period,  building 
simulations  from  the  procedures  and  executing  them  In  the  SOL 
as  a  "dress  rehearsal"  tor  the  formal  verification.  This  activity 
provides  verifiers  with  experience  In  using  the  SDL  test  bed  and 
also  serves  as  a  final  validation  of  the  SDL  capabilities  before 
FACI. 

Test  Documentation  and  Review 

Both  test  specifications  and  procedurea  are  centrally  docu¬ 
mented  In  an  Integrated  Test  Plan  (ITP).  For  STS-1  the  Verifica¬ 
tion  ITP  Is  comprised  of  10  books  The  first  eight  books 
document  the  detall/functlonal  testing.  Each  Is  associated  with  a 
major  software  area,  e.g.,  operating  system,  vehicle  checkout 
software,  guidance,  navigation,  and  control  software,  etc.  The 
ninth  book  documents  performance  testing.  These  first  nine 
books  cover  all  Verification  testing  and  are  reviewed  and 
approved  by  the  NASA  software  monitoring  organization,  the 
NASA  Spacecraft  Software  Division  (SSD),  under  the  authority 
of  the  Space  Shuttle  Orbiter  Project  Office.  Once  baselined,  the 
specifications  portion  of  these  books  cannot  be  changed  without 
the  formal  approval  of  NASA/SSD. 

The  content  of  the  tenth  book  grew  out  of  experience  gained 
during  Verification  of  the  Shuttle  Approach  and  Landing  Test 
software.  At  that  time  the  NASA  Space  Shuttle  Orbiter  Project 
Office,  along  with  representatives  from  the  avionics  engineering 
community,  reviewed  in  detail  the  entire  Verification  test  pro¬ 
gram  at  the  Configuration  Inspection  (Cl),  and  used  this  review 
as  a  basis  for  software  acceptance.  In  contrast,  the  plan  for 
STS-1  called  for  the  NASA  Project  Office  to  delegate  a  large  part 
of  this  review  to  NASA/SSD  as  an  ongoing  activity  and  to  review 
only  key  testing  areas  In  detail  at  the  Cl.  Book  10  of  the  ITP 
defines  this  key  subset  of  the  approximately  1100  verification 
test  cases.  This  subset,  which  Includes  all  of  the  performance 
cases  and  a  few  detall/functlonal  cases,  will  be  used  as  criteria 
for  software  acceptance  at  the  STS-1  Cl.  Selection  of  Cl  accep¬ 
tance  tests  has  been  coordinated  with  NASA  and  the  general 
Shuttle  engineering  community  over  the  past  31  months.  The 
content  has  been  baselined  and  under  NASA  Orbiter  Project 
Office  configuration  control  since  mld-197B. 

Formal  Verification  ■  Configuration  Control 

In  the  formal  verification  period,  FACI  to  Cl.  the  Verification 
organization  assumes  configuration  control  responsibility  lor 
both  the  software  and  simulator.  All  software  changes  Intro¬ 
duced  after  the  FACI  delivery,  both  discrepancies  and  require¬ 
ments  changes,  require  explicit  closure  by  the  Verification  area. 


Such  explicit  closure  sctlons  Include  re-executlon  of  test  esses 
(regression  testing),  execution  of  new  esses,  code  Inspections, 
lormsi  requirements  waivers,  or  explanatory  notes  to  software 
users. 

Configuration  control  of  software  discrepancies  (ORs)  starts 
with  all  ORs,  regardless  of  originating  facility,  being  maintained 
In  a  single  data  base  (see  Figure  8).  Open  discrepancies  are 
reviewed  weekly  by  a  committee  made  up  of  representatives 
from  all  IBM  software  organizations  and  NASA/SSD.  ORs  are 
dispositloned  In  one  of  three  ways:  a  decision  la  made  to  correct 
the  discrepancy:  the  DR  Is  not  considered  critical  to  the  STS-1 
mission  and  no  software  modification  Is  recommended  (a 
requirements  waiver  results):  or  the  OR  Is  determined  not  to  be  a 
software  problem.  The  latter  two  dispositions  are  made  only  with 
the  originator's  agreement.  Appeals  for  contested  dispositions 
are  resolved  either  by  NASA's  Test  &  Operations  Board,  to  be 
discussed  later,  or  the  NASA  Space  Shuttle  Orbiter  Project 
Office,  I.e.,  Orbiter  Avionics  Software  Control  Board  (OASCB). 

Following  disposition  and  implementation  (If  required),  ORs 
are  forwarded  to  the  Verification  group  for  Independent  re¬ 
assessment  and  formal  written  closure.  As  mentioned,  closure 
action  may  require  explicit  retest  or  coda  Inspection  regardless 
of  whether  a  software  modification  was  required.  Following  writ¬ 


ten  closure  of  ORs  by  Verification  each  DR  Is  coordinated  again 
with  NASA/SSD  or  the  NASA  Orbiter  Project  Office  for  formal 
concurrence. 

Explicit  closure  of  requirements  changes  Incorporated  after 
the  FACI  Is  similar  to  the  closure  process  for  DRs.  Approval  lor 
Implementation,  however.  Is  the  responsibility  oftheNASA  Orbi¬ 
ter  Project  Office. 


6.  FIELD  TEST  -  EARLY  CUSTOMER  INVOLVEMENT 

The  Ideal  software  development  cycle  discussed  earlier  (Fig¬ 
ure  4)  Is  only  realistic  when  applied  to  relatively  small,  well- 
defined  software  systems.  High  requirements  change  activity 
associated  with  developing  systems  and  schedule  restrictions 
make  It  especially  Impractical  whan  the  software  Is  one  of  many 
complex  systems  to  be  Integrated. 

Early  In  the  Shuttle  program  It  was  recognized  that  the  ultimate 
users  of  the  software  could  not  wait  for  the  complete  package  to 
be  developed,  verified,  and  delivered.  Since  the  software  was 
such  an  Integral  part  of  the  overall  system,  checkout  of  customer 
test  simulators  and  crew  trainers,  and  more  Important,  the  build¬ 
up  of  the  actual  Shuttle  orbiter  could  not  begin  without  It.  To 


Figure  8.  Software  Discrepancy  Closure  Procedure 
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minimize  the  elapsed  time  between  software  requirements  defi¬ 
nition  and  the  end  of  the  Shuttle  Integrated  testing,  a  series  of 
Interim  releases  of  the  software  to  the  customer  was  defined 

The  releasee  were  first  separated  by  flight  phase  or  memory 
configuration,  I.e..  ascent,  entry,  etc  .  staggered  In  time  by  sev¬ 
eral  months,  and  a  FACI  associated  with  each  Further  several 
early  "drops"  of  the  software  preceded  each  FACi  release,  allow 
Ing  a  buildup  of  capabilities  The  advantages  to  the  customer  are 
obvious,  but  this  approach  also  provides  certain  advantages  tu 
the  software  organization.  It  fosters  a  gradual  development  ano 
test  approach,  exposing  the  software  In  small  Increments  to  bottt 
verification  and  field  users.  This  allows  early  Identification  of 
software  discrepancies  and  eases  problem  Isolation  so  often 
difficult  In  large  fully-integrated  systems  This  incremental 
approach  also  exposed  the  software  and  data  proceaai  ng  system 
to  the  actual  flight  hardware  Interface  at  a  much  earlier  state  in 
the  program. 

The  STS-1  software  development  program  has  had  17  Interim 
releases  In  the  31  month  period  starting  In  October.  1677  (see 
Figure  6).  Although  full  software  capability  was  provided  after 
the  ninth  releaae  In  December  1978,  an  additional  eight  releases 
of  the  software  have  been  necesaary  to  accommodate  the  high 
requirements  change  and  discrepancy  activity  Inherent  In  large, 
complex,  first-of-a-kind  software  ayatems. 


Along  with  tha  advantagea  of  this  early  releaae  philosophy 
also  came  the  inevitable  disadvantages  of  configuration  control. 
Because  the  installation  of  new  software  releases  Into  field  sites 
la  not  instantaneous  or  without  cost,  there  Is  always  some  resis¬ 
tance  to  move  up  to  new  systems  as  they  become  available. 
Experience  has  shown  that  It  Is  not  unlikely  to  have  the  various 
Shuttle  field  sites  using  different  past  releases  of  the  software 
and  in  one  extreme,  a  single  site  using  multiple  releases  lor 
3‘lteient  types  of  testing  All  of  this  points  to  the  need  lor  a  strong 
Customer  Support  and  Field  Test  organization  with  a  mors 
expanded  role  than  that  described  by  the  Ideal  development 
cycle  of  Figure  4 

STS-f  Custom»r  Support  and  Flald  Tasf  - 
The  Tear  and  Oparallona  Group 

Test  and  Operations  (T&O)  Is  a  support  organization  which 
consists  of  four  Shuttle  test  site  resident  groups  and  a  staff  group 
In  Houston  Tha  Shutlls  test  sites  supported  are:  Flight  Systems 
Laboratory,  the  Shuttle  Avionics  Integration  Laboratory,  Shuttle 
Mission  Simulator,  and  Kennedy  Space  Center.  The  T &0  organi¬ 
zation  la  charged  with  six  basic  responsibilities:  user  training, 
assistance  In  the  preparation  and  review  of  site  test  procedures, 
flight  software  Installation  and  maintenance,  test  support,  mis¬ 
sion  support,  and  user  liaison  Each  of  these  responsibilities  will 
be  addressed  In  detail  and  their  value  to  software  development 
and  verification  will  be  Identified. 
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Figure  9.  Interim  Flight  Software  Releases 


site  user  training  Is  provided  In  two  steps.  First,  classes  at  the 
user  facility  cover  the  basic  design  and  Irnplementatlon  of  the 
flight  software.  The  second  step  Is  release  orientation.  For  each 
software  release,  training  sessions  familiarize  the  uaer  with  any 
new  capabilities  Included  In  the  release.  Other  topics  Include 
identification  of  changes  from  previous  releases,  and  known 
anomalies  and  restrictions. 

On  request,  T&O  personnel  assist  users  with  test  procedure 
preparation  and  review  for  both  the  vehicle  and  software.  This 
practice  ensures  that  the  flight  software  Is  used  properly,  the 
required  capabilities  are  available  In  the  release  being  used,  and 
alternate  approaches  are  recommended iwhen  the  desired  capa¬ 
bilities  are  not  available.  These  reviews  also  assist  the  Houston 
development  organizations  In  setting  software  development 
priorities  lor  each  Held  release. 

T&O  Installs  and  maintains  flight  software  releases  to  the 
degree  desired  by  the  site  management.  This  activity  Includea 
assistance  In  preparation  of  required  procedures  and  Installation 
of  releases  or  release  updates.  Throughout  this  activity  feedback 
Is  provided  to  the  software  development  organization.  Problema 
are  Identified  and  corrections  needed  to  support  vehicle  or  site 
testing  milestones  are  defined.  Requests  lor  software  fixes  are 
made  through  the  T &0  Board  -  a  NASA  luhctlon  which  controls 
flight  software  maintenance  to  user  facilities. 

As  part  of  the  pre-release  quality  assurance  process,  T&O 
performs  a  load  and  execution  test  on  all  flight  software  releases. 
These  pre-release  tests  are  performed  Ir,  at  least  one  field  site.  In 
conjunction  with  the  System  Integration  organization  men¬ 
tioned  earlier.  Site  test  support  requires  T &0  to  assist  the  sites  In 


conducting  tests  and  training  personnel  for  a  test  or  actual  mis¬ 
sion.  This  support  Includes  planning,  running,  and  analyzing 
tests.  T&O  personnel  accomplish  these  functions  by  monitoring 
the  site  tests,  and  analyzing  all  apparent  flight  software  anoma¬ 
lies.  Problems  Identified  In  the  software  are  directed  back  to 
Houston  lor  resolution  and  correction.  This  activity  provides  the 
Software  organization  with  additional  Insight  Into  the  behavior  of 
the  software  In  a  vehicle  environment  and  under  mission 
conditions. 

Mission  support  Is  provided  to  both  the  Mission  Control  Cen¬ 
ter  in  Houston  and  the  Launch  Control  Canter  at  Kennedy  Space 
Center  In  Florida.  T&O  personnel  support  vehicle  checkout  for 
launch,  launch  countdown,  and  throughout  the  mission  All 
flight  software  anomalies  detected  during  a  mission  are  docu¬ 
mented,  data  Is  gathered,  and  analysis  Is  performed  with  T&O 
assistance.  Additional  resources  needed  to  resolve  the  problem 
either  from  the  Software  organization  or  the  SOL  are  coordi¬ 
nated  by  T&O.  Following  major  tests  or  missions,  a  T&O  "quick- 
look"  report  summarizes  the  problems  encountered  and 
provides  problem  status.  After  all  problems  have  been  resolved, 
a  final  report  Is  written  to  Identify  the  closure  rationale.  Again, 
flight  software  prr^Llems  are  reported  to  Houston  for  correction 
or  final  disposition. 

T &0  personnel  act  as  a  liaison  between  users  of  flight  software 
at  the  sites  and  the  software  organizatloh  (see  Figure  10).  They 
also  provide  schedule  coordination,  assessment  of  priorities, 
problem  reports,  and  patch  coordihatlon  through  the  NASA  T&O 
Board.  The  T &0  Board  provides  final  resolutions  with  assistance 
from  the  Orbiter  Avionics  Software  Control  Board  (OASCB)  of 
the  NASA  Space  Shuttle  Orbiter  Project  Office. 
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Figure  10.  Test  end  Operations  Organization  Interfaces 
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Th«  T&O  functions  covered  thus  Isr  sre  genersl  responsibili¬ 
ties  thst  do  not  chenge  during  the  life  cycle  of  s  softwsre  deliv¬ 
ery.  However,  emphesee  does  shift  from  one  function  to  another 
as  the  program  progresaas.  The  software  life  cycle  can  be 
divided  Into  three  perloda:  pre-laat-source-bulld,  post-last- 
source-bulld,  and  poat-ConfIguratlon  Inspection. 

During  the  first  period,  slte-resldent  T&O  personnel  work 
closely  with  software  users  to  Identify  software  capabilities 
required  for  future  testa.  This  Information  Is  provided  to  the 
Software  Development  organization  to  be  factored  Into  the 
scheduling  of  software  capabilities.  Software  discrepancies  are 
also  reviewed  for  test  Impact  and  those  affecting  tests  are  Identi¬ 
fied  for  correction. 

The  emphasis  during  the  second  period  Is  primarily  on  mainte¬ 
nance.  Software  anomallee  that  Impact  testing  are  Identified  and 
patches  requested  through  the  T&O  Board.  Periodic  patch 
releases  consistent  with  site  test  plans  are  scheduled  for  deliv¬ 
ery.  During  this  period,  the  T &0  Board  controls  the  maintenance 
activity. 

The  post-CI  period  Is  controlled  by  the  Orbiter  Avionics  Soft¬ 
ware  Control  Board.  All  changes  requested  during  this  period 
must  be  approved  by  the  OASCB.  Requests  for  changes  are 
coordinated  through  the  T&O  Board,  and  taken  to  the  OASCB 


for  final  disposition.  Again,  the  major  emphasis  Is  on  finding  and 
Identifying  problems  rather  than  Identifying  needed  capabilities. 

7.  art-l  EXPERIENCE  TO  DATE 
Sonw»r»  Chtngt  Activity 

The  software  requirements  change  activity  Is  shown  In  Figure 
11.  The  number  of  change  requeats  Is  measured  from  the  pro¬ 
grammatic  baseline  of  the  various  requirements  documents.  As 
described  In  the  figure,  specific  requirements  books  were  base¬ 
lined  at  different  times. 

Formal  Verification  of  the  STS-1  software  began  In  early 
December  1978,  shortly  after  the  last  requirements  document 
was  baselined.  At  this  time  all  baseline  capabilities  necessary  to 
perform  the  STS-1  mission  were  Incorporated  In  a  single  soft¬ 
ware  release  (refer  to  Figure  9).  Eight  early  raleaaas  of  the  soft¬ 
ware,  distributed  over  a  14-month  period,  preceded  this  first 
"complete"  delivery. 

Since  this  delivery,  high  software  change  activity  has  necessi¬ 
tated  nine  additional  releases  of  the  software.  The  changes,  both 
requirements  updates  and  software  discrepancies,  are  expected 
In  auch  a  complex  first-of-a-kind  system.  Nearly  700  require¬ 
ments  change  requests  (CRs)  have  been  received  since 


Figure  11.  STS-1  Flight  Software  Requirements  Change  Requeats  (CRs) 
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Figure  12.  STS*1  Software  Discrepancy  Statistics 


D«c«mb«r  1978  (refer  to  Figure  11).  About  450  of  these  heve 
resulted  In  eoftwere  elteretlone;  the  remainder  are  documenta¬ 
tion  clarlficatlona.  The  changes  have  ranged  from  the  very  sim¬ 
ple,  luch  ae  changes  to  Initialization  constants  (l-LOAOs),  to  the 
very  complex,  e.g..  a  redesign  of  the  redundancy  management 
software  for  the  reaction  control  system. 

In  the  same  period,  approximately  3000  discrepancy  reports 
(DRs)  have  been  written  against  the  software.  Of  these,  52H  have 
resulted  in  changes  to  the  software  (see  Figure  1 2)  .  An  additional 
4%  were  found  to  be  software  problems  but  have  resulted  In 
requirements  waivers.  The  remainder  have  been  determined  not 
to  bo  problems  In  the  software.  As  previously  mentioned,  the 
Verification  organization  Is  responsible  for  explicit  re- 
asaosament  and  closure  of  aN  discrepancies  and  roquiromonts 
changes  whether  or  not  a  software  alteration  resulted. 

Current  V^fltlcutlon  Status 

At  this  time,  the  Verification  activity  Is  essentially  In  the  regres¬ 
sion  or  retest  portion  of  Its  test  plan.  As  of  Juno  1979,  the  detall- 
/f  unctlonal  tost  organization  had  completed  Its  base-plan,  I.e.,  alt 
cases  had  been  executed  and  analyzed  at  least  one  time.  The 
major  activities  since  Juno  have  been  to  retest  those  areas  which 
have  been  changed  duo  to  CRa  and  DRs,  and  to  complete  certain 
audits  and  surveys  associated  with  the  common-function  tests. 


The  performance  area  completed  Its  base-plan  as  of 
November  1979,  and  Is  also  in  a  regression  test  period.  The 
choice  of  tests  to  bo  rerun  In  the  performance  area  Is  driven  not 
only  by  CRs  and  DRs,  but  also  by  mission  timeline  updates, 
trajectory  changes,  and  crew  procedures  modifications. 

The  cost  of  a  total  Independent  Verification  program  com¬ 
pared  with  that  of  other  software  activities  Is  difficult  to  estimate 
because  of  the  many  overlapping  functions.  A  gross  estimate, 
however,  would  place  It  at  about  one-half  the  software  develop¬ 
ment  cost,  depending  on  the  level  of  testing  performed  within  the 
development  groups.  The  effectiveness  of  the  independent  veri¬ 
fication  of  the  Shuttle  software  can  be  measured  by  the  number 
of  problems  uncovered,  or,  from  another  perspective,  by  the 
number  of  problems  which  would  have  gone  undiscovered  If  the 
"Verification  Test  Plan"  had  been  executed  by  the  same  person¬ 
nel  reeponsible  for  the  software  development.  While  these  are 
difficult  numbers  to  interpret,  approximately  37%  of  the  total 
discrepancies  found  In  the  software  were  found  by  the  Verifica¬ 
tion  group.  Many  of  these  were  uncovered  In  off-nomlnal  situa¬ 
tions  and  generally  represent  real  system  problems.  Error-free 
software  Is  a  difficult  goal  to  achieve,  and  as  noted  earlier,  can 
only  bo  approached  asymptotically.  The  fraah  look  at  the  soft¬ 
ware  by  an  Independent  group  equally  familiar  with  both  the 
requirements  and  the  design  must  bring  the  project  closer  to  this 
goal.  Another  Indirect  measure  of  the  value  ot  the  Verification 
activity  can  bo  found  by  analyzing  the  fleld-site-generated  DRs/ 
corrections  shown  In  Figure  13.  From  these  graphs  It  Is  evident 
that  the  overall  quality  of  the  software  Is  Improving. 


Figure  13.  STS-1  Software  Discrepancy  Status  for 
Field  Sites 


•.  SUMMARY  -  AtSURINa  SOFTWARE  RELIASILITY 

The  following  outline  summerizee  the  key  factors  In  producing 
a  Shuttle  software  system  which  meets  program  requirements. 
All  major  software  activities  are  listed,  along  with  Items  consid¬ 
ered  Integral  to  their  success. 

Software  Dtvtiopment  and  Intagrallon 

e  Early  Involvement  In  the  development  of  the  soft¬ 
ware  requirements  through  a  dedicated  analysis 
organization. 

e  Adequate  documentation  of  the  requirements 
under  strict  configuration  control. 

e  Early  definition  of  programming  standards  and 
techniques,  and  enforcement  of  these  through  use 
of  a  high  order  language/compller  as  well  as  formal¬ 
ized  review  of  the  software. 

e  Centralized  configuration  control  of  the  teat  and 
build  of  the  evolving  system. 

e  A  thorough  but  limited  system  test  plan  to  assure 
that  the  Integrated  system  Is  usable  under  nominal 
conditions  before  delivery  for  formal  Verification 
or  for  the  customer's  early  use. 

Software  Varlficatlon 

e  Use  of  an  independent  line  organization  without 
managerial  or  personnel  ties  to  the  development 
organizations. 

e  Use  of  personnel  with  engineering  analysis  skills 
and  systema  experience. 

e  Maintenance  of  an  adversary  relationship  with  the 
developers  of  the  software,  I.e.,  the  assumption  that 
the  software  Is  essentially  untested  when  received. 

e  Sufficient  lead-time  to  develop  a  thorough  test  plan. 
This  includes  sufficient  time  for  Involvement  In  the 
software  requirements  definition  and  the  software 
design 

e  A  test  plan  which  provides  an  Independent  test 
against  requirements,  a  functional  level  of  testing 
and  appropriate  emphasis  on  code  surveys  and 
audits  to  address  problems  not  easily  uncovered  by 
testing. 

e  A  performance  or  system-level  test  plan  which 
emphasizes  software  user  (customer)  involvement 

at  all  stages. 

e  Configuration  control  of  test  documentation,  test 
execution  and  analysis. 

e  Configuration  control  of  changes  to  the  software 
test  article  along  with  a  formal  plan  for  re-test. 


Early  Customer  Uaa  ol  tha  Soflwara 

e  Delivery  of  early  (but  usable)  versions  of  the  soft¬ 
ware  to  the  customer  to  Increase  "shelf  life"  and 
provide  exposure  to  operational  flight  hardware. 

e  Use  of  a  capable  customer-sIte-resIdent  liaison  and 
maintenance  group  able  to  troubleshoot  problems 
and  provide  temporary  solutions. 

Simulator  -  Test  Bad 

e  Early  requirements  definition  Involving  all  future 

users. 

e  A  thorough  test  plan  which  Includes  a  final  valida¬ 
tion  by  users  of  the  facility  before  formal  software 
testing  begins. 

e  An  adequately-staffed  maintenance  and  test  group 
to  address  updates  and  discrepancies. 

e  Configuration  control  of  discrepancy  corrections 
and  updates  to  simulator  versions  being  used  for 
software  testing. 


9.  FUTURE  ACTIVITIES 

While  the  development  and  verification  of  tha  Shuttle  primary 
software  for  STS-1  poses  unique  challenges,  the  future  Involves 
many  unsolved  problems  associated  with  producing  "man¬ 
rated"  software  tor  the  high  flight  frequencies  projected  during 
tha  Shuttle's  operational  phase.  These  flights  will  require  rapid 
reconfiguration  to  accommodate  a  variety  of  payloads  and  tra¬ 
jectories.  This  will  cause  changes,  not  only  to  the  crew  and 
payload  support  software,  but  also  to  tha  guidance,  navigation, 
and  control  systems.  NASA  has  recognized  this  problem  and  is 
moving  forward  with  two  separate  activities.  The  first  Is  to  define 
a  Software  Production  Facility  (SPF),  which  will  serve  as  a 
follow-on  to  the  SDL  facility,  and  can  support  the  anticipated 
reconfiguration  activity.  This  includes  design  of  data  base  struc¬ 
tures,  evaluation  of  resource  requirements,  generation  of  oper¬ 
ating  procedures,  and  definition  and  development  of 
preprocessors  to  allow  rapid  reconfiguration  of  program  data. 

The  purpose  of  the  second  task  Is  to  reduce  the  test  time 
required  to  certify  the  software  for  flight.  The  first  objective  in 
this  activity  Is  to  define  and  develop  tools  to  aid  In  the  analysis  of 
simulation  test  data.  The  second  Is  to  develop  and  execute  a  test 
plan  to  stress  the  software  over  the  extremes  expected  during  the 
opsratloiHil  phase.  This  plan  will  Include  variations  In  mass  prop¬ 
erties,  orbital  Inclinations,  altitudes,  etc.,  and  will  provide  a  base 
set  of  test  cases  to  be  executed  on  the  actual  flight  software  load. 

The  definition  of  the  SPF  and  the  definition  ol  additional  analy¬ 
sis  tools  are  currently  underway.  Plans  are  In  place  to  begin  the 
stress  cese  definition  and  execution  in  the  third  quarter  of  1980. 


COMPUTERS  FOR  WEAPON  SYSTEMS  -- 
A  LOOK  AHEAD 

David  A.  Herrelko,  Captain,  USAF 

Since  our  national  defense  posture 
depends  more  on  superior  technology  than 
on  strength  of  numbers,  we  must  treat 
our  technology  advantage  as  a  precious 
resource,  to  be  preserved  and  extended. 

In  an  era  of  spectaculan  advances  it  is 
hard  to  measure  our  technology  "edge". 

In  a  time  when  the  purchasing  power  of 
the  Defense  dollar  is  shrinking,  it  will 
be  even  harder  to  keep  our  lead.  National 
security  rests  on  split-second  timing  and 
sure,  measured  control  of  military  power. 

A  key  factor  in  the  power  equations,  and 
a  critical  part  of  our  technology  lead, 
is  computational  capability  -  especially 
in  computers  embedded  in  weapon  systems. 

Computers  are  integral  to  virtually  every 
modern  military  system.  Almost  every¬ 
thing  that  flies  carries  embedded  com¬ 
puters,  from  satellite  to  bomber  to  our 
smallest  remotely-piloted  vehicle.  Mili¬ 
tary  surface  systems,  as  well,  rely  on 
computers.  These  rugged  digital  systems 
range  from  single-chip  processors  to  giant 
computer  centers  with  mainframe  and  the 
full  array  of  peripherals.  We  use  mili¬ 
tary  computers  in  laser  rangefinders, 
radar  signal  processors  and  electronic 
warfare  suites.  Our  embedded  computers 
perform  navigation,  weapon  delivery,  main¬ 
tenance,  battle  management,  and  a  host  of 
other  tasks.  These  new  capabilities  are 
expensive  and  hard  to  develop.  To  keep 
our  technology  lead  in  fielded  systems, 
we  must  manage  computer  technology  so 
these  systems  become,  not  cost  multipliers, 
but  force  multipliers. 

We've  all  heard  that  an  explosion  of  elec¬ 
tronic  technology  Is  underway,  and  it's 
true.  Although  explosions  are  hard  to 
manage,  we  should  welcome  this  technology 
explosion,  so  long  as  we  can  harness  part 
of  its  power  to  satisfy  real  mission  needs. 
The  Air  Force  Systems  Command  is  nearing 
the  completion  of  a  study  identifying 
computer  technologies  to  meet  weapon  sys¬ 
tems  needs  for  the  next  twenty  years.  The 
Computer  Technology  Forecast  and  Weapon 
Systems  Impact  St^y  (CdHTEC-2000)  defines 
policies  and  R  &  D  programs  that  will  cap¬ 
italize  on  commercial  advances  in  computer 
technology  and  also  satisfy  technology 


needs  unique  to  the  Air  Force.  The 
COMTEC-2000  study  indicates: 

0  Hardware  is  advancing  at  the 
speed  of  light. 

0  Software  is  proceeding  at  the 
speed  of  sound. 

0  Systems  understanding  evolves 
at  the  speed  of  human  thought. 

Hardware  Forecast: 

"Buck  Rogers"  projections  of  hardware 
technology  advances  all  begin  to  look 
the  same  after  a  while  -  every  forecast 
is  jammed  with  exponential  curves,  which 
I'll  spare  you  in  this  short  note.  A 
few  highlights  from  COMTEC-2000  are  worth 
special  attention: 

In  microcircuitry,  expect  a  thousandfold 
improvement  in  cost,  speed,  and  size  over 
the  next  decade.  A  single  chip  micro¬ 
processor  capable  of  a  million  instruc¬ 
tions  per  second  -  "a  MIP  on  a  chip"  - 
could  appear  in  as  soon  as  18  months.  By 
1985  we  should  see,  on  one  integrated  cir¬ 
cuit,  the  processing  capability  of  an 
IBM  370/158.  Bubble  memories  and  charge- 
coupled  devices  will  displace  electro¬ 
mechanical  bulk  memories,  especially  in 
space,  where  we  need  systems  that  will 
survive,  unattended,  for  five  years. 

Still  more  fantastic  potential  exists 
in  gallium  arsenide  and  Josephson  junc¬ 
tion  devices,  which  will  take  us  beyond 
the  limits  of  conventional  silicon  tech¬ 
nology.  Microprocessors  will  proliferate 
we  will  find  them  everywhere,  doing  tasks 
never  before  imagined.  The  challenge  to 
the  software  community  and  to  system  archi 
tects  is  to  capitalize  on  this  wealth  of 
hardware  technology. 

Software  Forecast: 

The  software  picture  for  the  next  twenty 
years  is  harder  to  project.  Unlike  hard¬ 
ware,  the  software  world  lacks  a  standard 
set  of  metrics  by  which  to  judge  perform¬ 
ance.  Even  the  vocabulary  in  the  field 
has  not  matured  to  a  standard  set  of  words 
and  meanings. 

No  quantum  leaps  are  forecast,  but  problem 
oriented  higher-order  languages  will  grow 
in  power  and  efficiency,  replacing  assem¬ 
bly  code  in  most  applications.  When  the 


Air  Force  selects  a  higher-order  language, 
logistics  considerations  will  dominate 
technical  ones,  as  increasing  emphasis  is 
placed  on  the  total  life  cycle  cost  of 
the  software  support  system.  Unlike  our 
experience  with  COBOL  in  the  1960's,  how¬ 
ever,  we  can't  expect  government  prefer¬ 
ence  for  one  higher  order  language  to 
guarantee  industry  acceptance. 


Air  Force  weapon  systems.  From  this  work, 
literally  hundreds  of  ideas  emerged.  The 
final  phase  of  the  study  focused  on  twelve 
key  issue  areas,  recommending  program  and 
policy  initiatives  to  exploit  opportuni¬ 
ties  and  to  overcome  potential  shortfalls. 
The  twelve  issue  areas  are; 

1.  Sol  icon  Technology 


There  will  be  a  software  personnel  short¬ 
fall  of  about  thirty  percent  nationwide, 
within  10  years.  Training  efforts  will 
intensify,  but  this  critical  shortage 
will  limit  DOD  industry.  Modern  program¬ 
ming  methods  will  double  productivity, 
and  interactive  systems  will  further  aid 
programmer  effectiveness.  "Automatic 
programming"  from  English  specifications 
to  machine  code  is  far  downstream,  but 
progress  will  continue  in  automating  the 
software  job. 


Systems  Forecast: 


The  COMTEC-2000  crystal  ball  is  most 
obscured  in  the  area  of  systems  forecast¬ 
ing,  and  this  is  disturbing.  Hardware 
breakthroughs  and  the  more  gradual  improve 
ments  in  software  engineering  cannot  be 
fully  exploited  until  we  develop  and  prac¬ 
tice  a  modern  approach  for  systems  design 
and  development.  If  the  maturity  of  a 
technology  is  marked  by  the  existence  of 
a  common  vocabulary  and  standards  of  per¬ 
formance,  the  systems  aspect  of  computer 
technology  must  be  our  weakest  area. 


There  will  be  system-level  changes,  driven 
by  the  hardware  explosion  and  software 
techniques.  For  combat  systems  where 
secure  communications  cannot  be  guaranteed, 
command,  control,  and  communications  (C^) 
and  aircraft  systems  will  need  methods  for 
multi-level  security,  fault  tolerance,  and 
graceful  degradation.  Giant  uniprocessor 
mainframes  will  yield,  in  many  applica¬ 
tions,  to  distributed  systems,  with  "smart" 
terminals  widely  used  in  stand-alone  or 
network  modes.  Conventional  hierarchies 
of  networked  machines  will  be  challenged 
by  peer-coupled  systems,  but  much  effort 
remains  before  we  understand  distributed 
data  bases  and  network  security. 


2.  Software  Personnel  Shortfalls 

3.  Software  Technology 

4.  Software  Life-Cycle  Investment 
Strategy 

5.  Distributed  Processing  Technology 
Needs 

6.  Technology  Exploitation  Time  Lag 

7.  Solution  Methodologies  Shortfalls 

8.  Computer  Resource  Standardization 

9.  Acquisition/Management  Approach 
for  C^  Systems 

10.  Nonsilicon  Technology 

11.  Man-Machine  Interface  Technology 

12.  Data  Generation  and  Handling. 

The  COMTEC-2000  report  is  available 
through  the  Defense  Technical  Information 
Center,  Cameron  Station,  Alexandria,  VA 
22314: 

Vol .  I  Summary 

AD  B  034954 

Vol.  II  Technical  Data 
AD  B  034955 

Vol.  Ill  Issue  Panel  Reports 
AD  B  041871. 


COMTEC-2000  Initiatives: 

The  COMTEC-2000  study  considered  these 
forecasts  and  measured  their  Impact  on 
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ABSTRACT 

An  introduction  to  the  need  for  software 
standardization  is  given,  and  the  role  of 
MIL-STD-1750  (USAF)  is  described  in  the 
context  of  the  proliferation  of  languages 
and  computer  architectures  used  in  weapon 
systems.  The  status  of  the  Air  Force's 
Airborne  Computer  Instruction  Set  Archi¬ 
tecture  (ISA)  is  reported,  and  a  technical 
overview  is  made  of  the  ISA  and  of  enhance¬ 
ments  to  it  proposed  by  the  industry-based 
MIL-STD-1750  Users  Group  to  the  government 
control  board.  Potential  applications  are 
described,  advantages  and  disadvantages  of 
the  approach  are  given,  and  challenges 
that  remain  are  identified. 

INTRODUCTION 

Since  our  national  defense  posture  depends 
more  on  superior  technology  than  on 
strength  of  numbers,  we  must  treat  our 
technology  advantages  as  a  precious 
resource,  to  be  preserved  and  extended. 

In  an  era  of  spectacular  advances  it  is 
hard  to  measure  our  technology  "edge." 

In  a  time  when  the  purchasing  power  of  the 
Defense  dollar  is  shrinking,  it  will  be 
even  harder  to  keep  our  lead.  National 
security  rests  on  split-second  timing  and 
sure,  measured  control  of  military  power. 

A  key  factor  in  the  power  equations,  and 
a  critical  part  of  our  technology  lead, 
is  computational  capability  -  especially 
in  computers  embedded  in  weapon  sys¬ 
tems  (1). 

Computers  are  integral  to  virtually  every 
modern  military  system.  Almost  everything 
that  flies  carries  embedded  computers, 
from  satellite  to  bomber  to  our  smallest 
remotely-piloted  vehicle.  Military  sur¬ 
face  systems,  as  well,  rely  on  computers. 


These  rugged  digital  systems  range  from 
fuel -savings  advisory  systems,  crash  data 
recorders  and  display  controllers  to  navi¬ 
gation/weapon  delivery  systems,  electronic 
warfare  (EW)  suites,  and  large  battle  man¬ 
agement  systems.  These  new  capabilities 
are  expensive  and  hard  to  develop.  To 
keep  our  technology  lead  in  fielded  sys¬ 
tems,  we  must  manage  computer  technology 
so  these  systems  become,  not  cost  multi¬ 
pliers,  but  force  multipliers. 

The  Air  Force  Systems  Command  has  recently 
completed  a  study  identifying  computer 
technologies  to  meet  weapon  system  needs 
for  the  next  twenty  years.  The  Computer 
Technology  Forecast  and  Weapon  Systems 
Impact  Study  (COMTEC-2000)  indicates: 

0  Hardware  is  advancing  at  the  speed 
of  light 

0  Software  is  proceeding  at  the 
speed  of  sound 

0  Systems  understanding  evolves  at 
the  speed  of  human  thought  (2). 

The  explosion  in  electronics  technology  - 
like  any  explosion  -  is  hard  to  manage, 
but  the  phenomenon  is  welcome,  so  long  as 
we  can  harness  a  portion  of  its  power  to 
satisfy  real  defense  needs.  All  the 
curves  of  technological  progress  show 
positive,  dynamic  growth  -  the  only  curve 
that  goes  the  wrong  way  is  the  cost  of 
software. 

SOFTWARE  COSTS 

The  Air  Force’s  software  costs  are  sky¬ 
rocketing,  The  demand  for  software  for 
weapon  systems  is  typified  by  Figure  1, 
which  shows  the  growth  trend  for  on-board 
avionics  memory  (3).  Any  errors  in  the 
chart  are  on  the  conservative  side:  many 
computers  are  deeply  embedded  within  sub¬ 
systems  and  many  systems  are  being  updated 
with  more  memory.  Similar  curves  exist 
for  command,  control  and  communica¬ 
tions  (C3),  radar  processing,  EW,  arma¬ 
ments,  and  space  applications. 

Software  costs  are  growing  in  a  relative 
sense,  as  well.  Figure  2  (derived  from 
Barry  Boehm's  famous  diagram)  suggests 
that  hardware  breakthroughs  and  expanding 
applications  of  computers  are  shifting 
computer  systems  costs  toward  software 
development  and  support  (4).  These  curves 


are  most  striking  in  their  impact  on  a 
labor-intensive  industry,  particularly  in 
the  context  of  a  severe  software  personnel 
shortage,  the  limited  prospects  for  improv¬ 
ing  programmer  productivity,  and  the  often 
overlooked  "hidden  costs"  of  software 
(Figure  3).  The  dramatic  growth  in  oper¬ 
ational  software  depicted  in  Figure  1  is 
just  the  tip  of  the  software  iceberg  (3). 
For  each  system  with  embedded  computer 
resources,  the  program  director  must 
manage  -  and  pay  for  -  an  entire  complex 
of  development  and  support  tools. 

The  Air  Force  Systems  Command  is  conduct-* 
ing  a  three-front  campaign  to  improve  the 
acquisition  and  support  of  embedded  com¬ 
puter  systems  through  competition,  tech¬ 
nology,  and  standardization.  These 
efforts,  when  guided  by  sound  management 
principles,  can  hold  the  line  on  costs, 
and  produce  better  fighting  systems. 

Competition  is  a  key  to  our  free  enterprise 
system.  No  single  initiative  can  do  more 
to  improve  the  cost-effectiveness  of  our 
military  systems.  Electronic  technology 
efforts  span  too  broad  a  spectrum  to  cover 
here,  but  a  few  are  particularly  relevant. 
The  Air  Force  is  concentrating  on  those 
areas  not  likely  to  be  advanced  by  the 
commercial  market  alone:  radiation  hard¬ 
ening,  packaging  for  severe  environments, 
fault-tolerant  computer  design,  and  very 
high  speed  integrated  circuits  are  relevant 
examples. 


iHO  im  tin  tin  imo 

Figure  1.  Memory  Requirements  Growth  in 
Initial  Versions  of  Avionics 
System 
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HARDWARE/  SOFTWARE  TRENDS 

Figure  2.  Computer  Systems  Life-Cycle 
Cost  Trends 


THE  SOFTWARE  ICEBERG 

HIDDEN  ELEMENTS  DF  SOFTWARE  SUPPORT 


Figure  3.  The  Software  Iceberg  --  Hidden 
Factors  in  Software  Support 


STANDARDIZATION 

Standardization  can  greatly  reduce  the 
needless  proliferation  of  computers, 
architectures  and  languages  in  the  military 
inventory  (5),  but  great  care  must  be  taken 
so  that  standardization  efforts  do  not  pre¬ 
vent  competition  or  discourage  innovation. 
In  order  to  exploit  the  best  hardware 
breakthroughs  and  to  permit  all  sources  to 
compete  for  military  business,  emphasis 
should  be  placed  on  standards  for  computer 
languages,  architectures,  and  interfaces. 
Observations  (e.g.,  6,  7)  of  the  literally 
hundreds  of  computers  and  languages  used 
in  military  systems,  and  the  resultant 
"M  X  N"  problem  (Figure  4)  of  proliferat¬ 
ing  combinations  of  languages  and  target 
architectures  have  led  to  DOD  Directive 
5000.29,  which  fosters  the  use  of  higher- 


order  languages  (HOL),  and  DOD  Instruction 
5000.21,  which  seeks  to  bound  the  set  of 
HOLS  used  1n  embedded  computer  systems 
(8,  9).  The  Instruction  Set  Architecture 
(ISA)  Standardization  Panel  of  the  Office 
of  the  Secretary  of  Defense  Management 
Steering  Committee  for  Embedded  Computer 
Resources  has  drafted  and  recommended  the 
publication  of  DOD  Instruction  5000. 5X. 

The  thrust  of  this  new  instruction  (10) 
is  to  identify  and  foster  the  use  of  an 
approved  set  of  vendor-independent  ISAs 
which  may  be  implemented  with  the  latest 
hardware  technology.  This  approach  avoids 
the  technology  freeze  often  encountered 
with  "black  box"  standardization 
approaches,  and  still  permits  additional 
levels  of  standardization  (interfaces, 
F-cubed,  or  piece-part)  where  they  are 
appropriate  and  necessary.  An  example 
of  such  a  government-owned  ISA  is  MIL- 
STD-1750  (USAF),  "Airborne  Computer 
Instruction  Set  Architecture"  (11). 

MIL-STD-1750 

MIL-STD-1750  recently  celebrated  its  first 
birthday  as  the  Air  Force  standard  ISA. 

Born  at  Wright-Patterson  Air  Force  Base 
as  part  of  the  Dioital  Avionics  Informa¬ 
tion  System  (DAIS)  program,  1750  evolved 
through  successive  design  iterations, 
under  the  guidance  of  Dr.  Donald  Moon, 

Dr.  Mark  Michael  of  the  Air  Force  Avionics 
Laboratory  (AFAL),  and  Mr.  Joseph  Gebele 
of  the  Aeronautical  Systems  Division  (ASD). 
Early  work  produced  the  AN/AYK-15  DAIS 
processor.  This  computer's  ISA  was  evalu¬ 
ated  and  enhanced  (12,  13)  and  MIL-STD- 
1750  (USAF)  emerged.  1750  has  since  been 
used  to  build  the  AN/AYK-15A  computers  (14). 

MIL-STD-1750  (USAF)  defines  a  general  pur¬ 
pose  digital  computer  architecture,  as 
seen  by  a  compiler  or  by  an  assembly  lan¬ 
guage  or  machine  language  programmer. 

1750  prescribes  data  and  instruction  for¬ 
mats,  computer  organization  and  operation 
at  the  level  necessary  to  exploit  a  common 
base  of  software  support  resources  -  hard¬ 
ware,  software,  and  people.  The  standard 
is  not  a  single  computer  design,  a  form- 
fit  function  specification,  or  the  draw¬ 
ings  for  one  vendor's  "black  box". 

MIL-STD-1750  is  Intended  to  control  the 
proliferation  of  computer  designs  in  our 
weapon  systems.  Use  of  the  ISA  standard 
can  reduce  risk  for  program  managers. 
Government  and  contractors  alike  will  be 


able  to  exploit  a  growing  base  of  people, 
computer  programs,  and  support  facili¬ 
ties  -  developed,  validated  and  paid  for 
before  any  program  commitments  are  made. 

The  program  manager  can  initiate  software 
development  well  before  the  actual  mission 
computers  are  delivered.  Most  importantly, 
1750  will  focus  competition  on  a  proven 
ISA,  owned  by  the  Air  Force  and  available 
to  all  bidders. 

MIL-STD-1750  is  strongest  when  used  as 
part  of  the  Air  Force  Systems  Command's 
triad  of  embedded  computer  system 
standards: 

0  MIL-STD-1553B,  our  multiplex  bus 
standard  (15)  is  used  in  Air  Force, 
Army,  and  Navy  systems,  and  enjoys 
wide  use  internationally 

0  MIL-STD-1589A  (USAF),  "JOVIAL 
(J73)''.  is  the  Air  Force's  HOL- 
of-choice  for  embedded  computer 
systems  (16,  17,  18,  19) 

0  MIL-STD-1750  (USAF),  although  the 
"new  kid  on  the  block"  in  terms 
of  date  of  publication,  has 
evolved  over  five  years,  in  open 
forum  with  industry. 


Figure  4.  DOD  Instructions  Bound  the 
M  X  N  Problem 
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MIL-$TD-1750  STATUS 

How  1s  MIL-STD-1750  doing?  It  works  - 
and  works  well!  The  study  by  Dietz  and 
Szewerenko  (20)  reports  that  MIL-STD-1750 
outperformed  the  ISAs  of  the  AN/AYK-14, 
Digital  Equipment  Corporation  PDP-11/70, 
and  Data  General  Eclipse  C/330  in  terms 
of  memory  activity  and  program  size,  and 
that  1750  was  comparable  to  these  ISAs  in 
terms  of  processor  activity.  Bradley  (21) 
studied  1750' s  potential  for  electronic 
warfare,  and  found  that  the  1750  ISA  can 
support  EW  signal  processing  and  that 
1750 's  ISA  would  outperform  the  ISAs  now 
used  in  those  systems.  Denton,  et  al  (22) 
found  1750  ISA  basically  sound,  but  stated 
that  while  MIL-STD-1750  compares  favorably 
with  other  airborne  computer  architectures, 
several  "rough  edges"  remain.  Denton  and 
Samuelson  (23)  proposed  modifications  and 
extensions  to  1750,  which  they  believe 
would  Improve  overall  cost  and  performance 
by  twenty  to  forty  percent. 

Four  implementations  of  1750  have  already 
been  built,  in  the  Air  Force  Institute  of 
Technology,  for  AFAL,  and  for  ASD.  A 
total  of  thirteen  computer  manufacturers 
and  avionics  systems  houses  are  develop¬ 
ing  1750-based  computers.  Many  of  these 
firms  see  the  opportunity  to  reduce 
production  costs  of  their  existing  fam¬ 
ilies  of  computers  through  microprogram 
emulation  of  MIL-STD-1750.  Extending  the 
production  runs  of  already  deployed  com¬ 
puters  will  exploit  economics  of  scale, 
and  vendors  will  ride  the  production 
learning  curve  to  lower  unit  costs.  This, 
in  turn,  will  reduce  hardware  life  cycle 
cost  in  a  global  sense,  for  the  Air  Force, 
and  for  any  organization  buying  from  these 
military  computer  families.  Cutting  the 
acquisition  costs  of  computers  that  may 
not  use  the  1750  ISA  was  not  the  original 
goal  of  the  DAIS  program,  but  it  is  a 
most  desirable  spin-off.  The  most  common 
technology  for  early  Implementations  of 
1750  is  a  multiple-card  bi-polar  bit- 
slice  emulation,  but  other  approaches  are 
attracting  attention.  Vendors  may  exploit 
a  variety  of  technologies  in  building  a 
1750-based  computer.  For  weapon  systems 
applications,  low-power  I^L  using  gate 
arrays,  radiation-hard  CMOS/SOS,  and 
compact  very  high  speed  integrated  cir¬ 
cuit  (VHSIC)  approaches  are  attracting 
attention. 


SUPPORT  CONSIDERATIONS 

Natural  concerns  for  an  emerging  ISA 
standard  include  issues  of  certification 
and  software  support.  ASD's  Systems 
Engineering  Avionics  Facility  (SEAFAC), 
which  support  military  standards  require¬ 
ments  and  in-house  hot  bench  activities 
(24),  is  the  MIL-STD-1750  Control  Facil¬ 
ity.  SEAFAC  is  sponsoring  two  studies  of 
how  a  rigorous  certification  capability 
can  best  be  provided.  While  a  provisional 
acceptance  test  program  may  be  said  to 
exist,  SEAFAC  plans  a  more  robust  certi¬ 
fication  capability.  The  results  of  the 
certification  studies  will  be  presented 
at  the  next  quarterly  meeting  of  the  MIL- 
STD-1750  Users  Group,  and  industry  com¬ 
ments  will  be  solicited. 

Developing  a  mature  set  of  software  sup¬ 
port  tools  is  essential  to  successful  ISA 
standardization,  and  work  to  this  end  is 
well  under  way.  Early  parallel  efforts 
have  produced  a  variety  of  assemblers, 
simulators,  and  linkers/loaders  that  run 
on  a  variety  of  hosts.  A  formal  course 
is  one  set  of  tools  that  has  already  been 
completed.  AFAL,  ASD,  and  the  several 
contractors  involved  are  focusing  on  a 
Cownon  File  Format  that  should  ease  the 
hoped-for  eventual  convergence  to  a 
common  set  of  tools,  government-owned, 
and  available  to  all  through  the  Federal 
Software  Exchange.  Two  separate  J73  com¬ 
piler  developments  targeted  to  1750  are 
under  way.  The  MIL-STD-1750  Users  Group 
Software  Tools  Committee  is  continually 
updating  an  annotated  catalog  of  the  tools 
available  (25). 

MIL-STD-1750  USERS  GROUP 

The  MIL-STD-1750  Users  Group  is  a  highly 
successful  synergy  of  talent  from  industry 
and  government.  The  cream  of  industry's 
embedded  computer  architects  has  joined 
with  compiler  experts  and  systems  houses, 
and  they  are  complemented  by  government 
representatives  from  the  research,  acqui¬ 
sition,  logistics,  and  operational  com¬ 
mands  of  the  Air  Force.  The  users  group 
was  formed  in  August  1979,  and  has  been  a 
valued  contributor  in  the  refinement  of 
MIL-STD-1750A.  The  group's  energetic  - 
and  expert  -  technical  review  of  the  stan¬ 
dard  has  greatly  improved  the  proposed 
MIL-STD-1750A,  which  we  expect  will  be 
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our  standard  for  years  to  come.  Member¬ 
ship  in  the  group  is  open  to  all  comers; 
the  only  dues  asked  are  an  open  mind  and 
a  willingness  to  participate  on  one  of 
the  committees  shown  in  Figure  5.  Having 
greatly  aided  in  refining  MIL-STD-1750A, 
the  Users  Group  is  shifting  its  emphasis 
to  matters  of  support  tools  and  certifi¬ 
cation  methods.  A  typical  meeting  is 
attended  by  80  or  more  people,  represent¬ 
ing  dozens  of  companies.  Inquiries  may 
be  addressed  to  the  junior  author,  who  is 
Secretary  of  the  group. 

MIL-STD-1750  CONTROL  STRUCTURE 

Figure  6  depicts  the  government  control 
structure.  The  Deputy  for  Avionics  Con¬ 
trol  (a  Joint  AFSC/AFLC  organization)  is 
the  MIL-STD-1750  Control  Agent,  and  serves 
ex  officio  as  Chairman  of  the  Control 
Board.  The  Control  Board  is  composed  of 
government  representatives  from  labora¬ 
tories,  System  Program  Offices,  Air  Logis¬ 
tics  Centers,  and  the  using  commands,  as 
well  as  ASD  enigneers.  Participation  is 
open  to  other  military  organizations  hav¬ 
ing  a  stake  in  the  standard;  participa¬ 
tion  by  other  Services  is  encouraged. 

The  Board  is  charged  with  reviewing  all 
proposed  changes  to  the  standard,  and 
making  recommendations  on  technical  and 
management  matters  to  Headquarters,  Air 
Force  Systems  Command  (HQ  AFSC).  The 
immediate  task  of  the  Control  Board  is 
to  evaluate  the  change  pages  that  are 
proposed  for  the  enhanced  MIL-STD-1750A 
(USAF),  and  to  recommend  a  course  of 
action  to  HQ  AFSC.  By  the  publication 
date  of  this  paper,  we  hope  that  MIL-STD- 
1750A  (USAF)  will  have  been  released  for 
publication. 

TECHNICAL  OVERVIEW 

MIL-STD-1750  (USAF),  21  February  1979,  is 
basically  a  "16-bit"  ISA,  with  16  user- 
accessible  general  registers.  A  rich 
complement  of  16-  and  32-bit  instructions 
are  provided,  with  referencing  modes 
including:  instruction  counter  relative, 
memory  direct,  and  memory  indirect.  The 
Importance  of  the  stack  as  a  data  struc¬ 
ture  is  recognized  by  PUSH  and  POP  instruc¬ 
tions  which  access  an  implicit  stack 
pointer.  Any  of  the  general  registers 
may  be  used  as  a  return  stack  pointer,  in 
recognition  of  the  need  for  a  separate 
stack  for  return  linkage. 


MIL-STD-1750  USERS  GROUP 


Figure  5.  The  MIL-STD-1750  Users  Group 
Organization 


MIL-STD-1750  CONTROL  STRUCTURE 


Figure  6.  The  MIL-STD-1750  Control 
Structure 


Six  data  types  are  recognized:  bits, 
bytes,  16-  and  32-b1t  fixed  point,  32- 
and  48-bit  floating  point.  Data  referenc¬ 
ing  modes  Include:  register  direct,  mem¬ 
ory  direct  (indexible),  memory  Indirect 
(pre-indexible) ,  immediate  long,  immedi¬ 
ate  short,  base  relative  (0  to  255),  and 
base  relative  indexible.  Of  course,  not 
all  referencing  modes  apply  to  all  opera¬ 
tions.  The  short-format  base-relative 
addressing  facilitates  subroutine  argument 
and  random  array  element  access.  The 
base-relative  Indexed  addressing  eases 
Indexing  through  an  array  in  a  subroutine 


when  the  array  address  is  passed  as  an 
argument.  The  short-  and  long-format 
immediate  addressing  modes  make  the  use 
of  constants  easy. 

An  excellent  complement  of  shift  instruc¬ 
tions  is  given,  including  variable-length 
shifts  with  the  count  contained  in  a 
register.  A  special  branch  instruction 
exists  for  closing  loops,  and  a  separate 
interrupt  vector  (new  IC,  new  SD,  new  IM) 
is  used  for  each  interrupt  source. 

It  is  this  architectural  depth  and  breadth 
which  make  MIL-STD-1750  the  basically 
sound  design  that  it  is.  Several  aspects 
of  the  ISA  as  first  issued  have  been  iden¬ 
tified  in  Users  Group  committee  work,  how¬ 
ever,  that  made  refinement  of  the  ISA  an 
important  task.  Many  more  technical 
issues  were  raised  in  the  Standards  and 
Architecture  Committees  than  could  be 
resolved  in  the  time  permitted.  The  Air 
Force  recognized  that  a  good,  workable 
military  standard  "on  the  street"  was  far 
more  useful  than  an  almost-perfect  one 
that  was  constantly  being  refined.  Due 
to  pressing  program  needs,  the  Air  Force 
representatives  asked  the  Users  Group  to 
respond  to  a  15  February  1980  deadline  for 
recommendations  on  the  new  standard. 
Accordingly,  the  Users  Group  tackled  the 
usual  ambiguities  and  typographical  errors, 
and  also  recommended  enhancements  in 
selected  areas  of  the  architecture.  Due 
to  the  time  constraint  several  interesting 
problem  areas  could  not  be  pursued  in  depth 
and  many  technically  valuable  proposals 
could  not  be  incorporated  in  the  Users 
Group  recommendations.  The  diverse  back¬ 
grounds  of  the  participants  in  the  Users 
Group  discussions  caused  the  ISA  and  pro¬ 
posed  changes  to  be  scrutinized  from  vari¬ 
ous  points  of  view.  Issues  raised  included: 

0  efficiencies  of  hardware/software 
implementation  and  operation 

0  relative  ease  of  software  implemen¬ 
tation  and  checkout 

0  relative  ease  of  use  of  instructions 

0  life  cycle  costs 

Change  proposals  exemplary  of  those  the 
Users  Group  has  made  to  the  Control  Board 
include: 


0  definition  of  processor  Reset  state 
and  Power  Up  state 

0  addition  of  paging  scheme  for  mem¬ 
ory  expansion  (from  64k  words  to 
one  million  words) 

0  addition  of  privileged  instruction 
feature 

0  addition  of  extended-capability 
I  0  instructions 

0  definition  of  operation  of  Timers 
A  and  B 

0  clarification  of  operation  of 
"Clear  Fault  Register"  I/O 
instruction 

0  reassignment  of  base-registers 
and  reversal  of  original  stack 
pointer  arithmetic 

The  standard  is  by  no  means  the  ultimate 
in  computer  architectures  -  it  was  not 
intended  to  be.  While  many  changes  that 
might  have  made  1750A  technically  better 
did  not  get  Users  Group  endorsement,  those 
that  did  emerge  will  do  much  to  enhance 
the  already  broad  appeal  of  1750/1750A. 

POTENTIAL  FUTURE  APPLICATIONS 

The  1750/1750A  ISA  can  contribute  to 
reduced  risk  and  cost  in  a  wide  variety 
of  military  applications,  whether  airborne, 
ground-  or  space-based.  Potential  appli-  . 
cations  include: 

0  fuel  savings  advisory  systems 

0  optical  sensor  processing/control 

0  navigation 

0  stores  management  and  weapon 
delivery 

0  fuel  flow  and  center-of-gravity 
control 

0  controls/display  management 

0  electronic  warfare 

0  encryption/decryption 

0  radio  frequency  controllers 
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auto-land  systems 
radar  processors 
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0 

0  engine  malfunction  detection  and 
diagnosis 

0  automatic  test  equipment 
0  intelligence  processing 
0  battle  management  systems 
0  command,  control,  and  communications 
0  terminal  guidance  and  fuzing  control 
0  data  compression 

0  digital  voice  synthesis/recognition 

ADVANTAGES  AND  DISADVANTAGES 

There  are  good  reasons  to  use  1750/1750A. 

It  is  vendor  independent,  and  is  not  tied 
to  any  hardware  specifications.  Program 
managers  will  be  able  to  rely  on  a  set  of 
proven  support  tools  -  schedules  will  be 
shorter,  costs  lower,  and  we'll  have  fewer 
development  "surprises."  The  Air  Force 
can  save  millions  across  the  life  cycles 
of  many  systems.  1750/1750A  will  let  us 
exploit  the  hardware  technology  explosion 
without  "locking  in"  on  a  single  box. 
Competition  will  be  wide  open  -  to  all 
bidders. 

Concerns  about  1750  are  real,  however.,  It 
could  be  better,  but  time  constraints  did 
not  permit  it.  The  ISA  is  a  fairly  new 
design,  but  it  has  been  tested,  proven 
and  is  well -understood  by  the  industry. 

Our  support  software  is  several  months 
away,  but  once  it  is  "in  place,"  all  will 
benefit  from  it.  The  Air  Force  must  fund 
and  man  a  control  structure  and  validation 
facility,  but  the  costs  of  continued  pro¬ 
liferation  of  ISAs  is  far  greater  than 
this.  As  with  any  standard,  1750/1750A  is 
another  constraint  on  the  design  process  - 
but  we  think  it  a  helpful  one,  if  common 
sense  is  used.  1750/1750A  is  not  suited 
for  all  computing  tasks,  nor  was  it 
intended  to  be;  it  should  be  used  only  in 
those  portions  of  the  spectrum  of  embedded 
computer  applications  where  it  can  perfonn 
well.  Still,  1750/1750A  can  perform  many 
jobs  better  to  best. 


The  emerging  Air  Force  pattern  indicates 
that,  unless  there  are  compelling  technical 
and  life  cycle  cost  reasons  to  the  contrary, 
1750/1750A  shall  be  used  to  the  maximum 
extent  possible  in  airborne  embedded  com¬ 
puter  systems.  All  indications  are  that 
ground,  space,  and  armament  systems  will 
receive  similar  guidance  soon.  The  great¬ 
est  difficulty  to  overcome  in  fielding  any 
new  standard  is  that  no  program  manager  or 
vendor  wants  to  be  first.  The  multiplex 
bus  standard  (15536)  encountered  the  same 
early  "friction  loss",  and  has  become  an 
international  success.  MlL-STD-1750  has 
overcome  its  first  hurdles,  and  has  high 
potential  for  success. 

The  essential  factors  remaining  to  ensure 
that  success  are: 

0  a  broad  enough  scope  of 
applicability 

0  good  interrelationships  with 
related  standardization  efforts 

0  expectations  of  stability, 
balanced  against 

0  assurances  that  major  concerns 
will  be  heard 

0  a  broad  base  of  support  in  the 
community  that  uses  computers 

0  a  demonstrated  reduction  of  the 
total  life  cycle  cost  associated 
with  embedded  computing. 

This  is  no  simple  shopping  list,  in  fact, 
it  would  appear  to  pose  a  significant 
challenge  to  us  all.  The  continued  com¬ 
munication  and  cooperation  of  members  of 
industry  and  the  Air  Force  are  absolutely 
vital.  Here's  hoping  we  succeed. 
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SUMMARY  OF  ISSUES 
AND  RECOMMENDATIONS 


COMPUTER  SOFTWARE 


3.  Award  Fee  for  Software 


SUMMARY  OF 

ISSUES  AND  RECOMMENDATIONS 

1.  Software  Compliance  Documents 
Issue 

On  Air  Force  contracts  there 
currently  is  no  software  contract- 
ural  compliance  document  which 
provide  the  customer  control  over 
the  software  development  process/ 
methodology. 

Recommendations 

o  Establish  DID  for  Software  Man¬ 
agement  Plan  (use  NASA  plan  or 
example) . 

o Specify  submittal/ compliance 
requirement  in  RFP. 

2.  Standardization  Initiatives 
Issue 

There  is  a  high  potential  for  re¬ 
duced  life  cycle  costs,  less  pro¬ 
gram  risk,  shorter  development 
schedules  and  improved  system 
support^  through  standardization 
initiatives.  If  standards  are  ven¬ 
dor  and  technology  independent. 
Standard  High  Order  Languages 
(HOL),  computer  set  architectures 
multiplex  data  buses  and  common 
support  systems  can  significantly 
improve  our  way  of  doing  busi¬ 
ness  if  developed  and  applied 
using  common  sense  in  an  open 
forum  with  industry. 

Recommendations 

oSD  sponsor  a  study  with  a  recog¬ 
nized  association  to  determine 
applicability/exceptions  of 
MIL-STD-1750A  ISA 
MIL-STD- 1589A  Jovial  (J7 3) 
MIL-STD-1553B  Mux  Bus 
Common  system  support 
facilities,  industry  critique  and 
participation,  and  SD  policy 
established. 


Issue 

Award  fee  contracts  are  useful  in 
controlling /rewarding  excellence 
in  software  development  and 
management. 

Recommendations 

o  Develop  award  fee  contract 
provisions  emphasizing  soft¬ 
ware  computer  system  disci¬ 
plines. 

-  design  reviews 

-  configuration  management 

-  quality  assurance 

-  support  systems  development 

-  documentation 

-  performance 

-  timing  &  sizing 

-  growth 

o  Encourage  their  use. 

o  Develop  evaluation  tools  for  pro¬ 
curement  use. 

4.  Software  Quality  Assurance 
Issue 

MIL-S-52779A  has  not  produced 
the  desired  result; 

-Contractor  QA  personnel  not 
adequately  trained  in  software. 
-Procedures  for  software  quality 
assurance  program  implementa¬ 
tion,  inadequate  or  non-existant. 
-Software  test  tools  not  properly 
controlled. 

-Lack  or  inadequacy  of  tailoring 
and  enforcing  of  MIL-S-52779A 
by  the  government. 

Recommendations 

o  Establish  SD  contractor  qualifi¬ 
cation  survey: 

-  Develop  qualification  criteria. 

-  Review  contractor  implemen¬ 
tation  procedures  against  cri¬ 
teria. 

-  Assess  qualification  level  of 
SQA  personnel. 


-  Adequacy  of  control  of  tools. 

o  Develop  tailoring  guides  for  MIL- 
S-52779A. 

5.  AFPRO  Involvement 
Issue 

AFPRO  role  in  sof^are  develop¬ 
ment/test/support  unclear,  vis- 
a-vis  SPO. 

AFPRO  manning  and  training 
appear  inadequate  to  meaningfully 
affect  the  software  process  or 
product. 

Recommendations 

oSD  and  AFCMD  explore  roles/ 
mission  re;  SW  management. 

o  Establish  "master  MOA" 
(SD-AFCMD) 

-  state  desired  roles/missions 

-  outline  plan  to  match  AFCMD 
manning  and  training  with 
planned  duties. 

o  Individual  SPO/AFPRO  MOAs 

-  reflect  intent  of  "master  MOA" 

-  recognize  current  AFPRO 
capabilities 

-  adjust  roles/missions  accord¬ 
ingly 

o  Provide  funding  support  for 
AFPRO  training. 

6.  Software  Skills 
Issue 

AFSC  and  SD  critically  short  of 
competent  aerospace  software 
engineers /managers. 

Problem  getting  worse  -  fast. 

Recommendations 

o Improve  hygiene  factors 

-  pay 

-  housing 

-  promotions 


o  T  rain  what  you  have . 

o  Institute  AFSC/AFLC  "job  swap" 
program  to  prevent  misassign- 
ment  loss  of  specialists. 

o Promote  civilians  to  software 
management  jobs. 

oincrease  Aerospace  role. 

7.  Subcontractor  Software 
Issue 

Problems  in  many  cases  have 
arisen  from  inadequate  manage¬ 
ment  of  subcontracted  software. 

Recommendations 

o  Prior  to  contract  award: 

•  Ask  officers  to  detail  agree¬ 
ments  prior  to  award, 

-  Use  pre-award  survey  for 
proposed  subcontractors. 

-  Attend  subcontract  awards. 

8.  Software  Design  Reviews 
Issue 

MlL-STD-1521  is  too  general. 

Recommendations 

o Develop  criteria  for  tailoring 
MIL-STD-1521  for  computer 
software. 

9.  Lack  of  Computer  Specs. 

Issue 

Specifications  and  Data  Item 
Descriptions  (DIDs)  do  not  adequat- 
ly  address  requirements  for  very 
high  speed  computer  systems 
(pipeline /parallel/ signed  process¬ 
ors). 

Recommendations 

o  Working  group  to  evaluate  applic¬ 
ability  of  Specs  and  DIDs  to  very 
high  speed  computing  systems. 
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o  Recommend  new  or  modified 

Specs/DIDs. 


10.  Procurement  Practices 
Issue 


Contractor  automation  of  the  soft 
ware  development  process  allows 
production  of  cost  effective  docu¬ 
mentation.  SD  procurement  pra- 
tices  does  not  allow  alternates  to 
be  proposed  which  could  reduce 
costs  and  at  the  same  time  be 
more  effective  technically. 

Recommendations 

o  Extend  the  concepts  of  MIL-S- 
83490,  level  c  and  SAMSO-STD- 
73-2c  (on  parts  documentation 
savings)  to  software  and  develojb 
the  appropriate  contractural 
language.  Implement. 
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REQUEST  FOR  PROPOSAL  COMMUNICATIONS 


Chairmen 

Coordinator 

Colonel  Jim  Russell 

Dan  Browne 

Director,  Defense  Systems 

Contracts,  AF  Space  Division 

AF  Space  Division 

Neil  Lamb, 

Procurement  Analyst 

HQ  NASA 

Dick  Pooch, 

Director  Acquisition  Management, 

Space  Systems  Division 

Lockheed  Missile  And  Space  Co. 

WORKSHOP  PAPERS 

Precontract  Communications 

Lt  Col  Ken  Juvette,  AFSD 

Draft  SOW/DRFP  Effectiveness 

Mr.  Dick  Pooch,  LMSC  SSD 

Mr.  Bob  Crossley,  AFSD 

RFP  Preparation 

Mr.  Bob  Cook,  AFSD 

Mr.  Jerry  Madden,  NASA  GSFC 

Proposal  Preparation 

Mr.  Lou  Gomberg,  RCA  ASTRO 

Proposal  Preparation 

Mr.  Tom  Morris,  Rl  SD 

Proposal  Preparation 

Dr.  Sam  Silverberg,  HAC 

Proposal  Evaluation 

Mr.  Joe  Krueger,  AFSD 

Proposal  Evaluation 

Mr.  Clen  Neehan,  HQ  NASA 

PRECONTRACT  COMMUNICATIONS 
Lt.  Col.  Ken  Juvette,  AFSD 


SUBJECT:  A  CASE  STUDY  OF  PRECONTRACT 
INTERFACES 


QUESTION:  DOES  THIS  APPROACH  CONTRIBUTE 
TO  OVERALL  MISSION  ASSURANCE? 


CASE  STUDY-GPS  OPERATIONAL  CONTROL 
SYSTEM  SEGMENT 

•  SELECTED  SMALI  (I  S  PEOPLE)  DEDICATED  PROOflAM 

OFFICE  TEAM  SEP 

•  SYNOP5IZED  FOR  POTENTIAL  PRIMES  SEP  77 

•  CONDUCTED  CLOSED  BUSINESS  STRATEGY  FORUM  FOR 

CANDIDATE  PRIMES  NOV  77 

•  DEVELOPED  GOVERNMENT  BUSINESS  STRATEGY  DEC  77 

•  CONDUCTED  OPEN  FORUM  FOR  ALL  INDUSTRY  AND  RELEASED 
GOVERNMENT  S  REQUIREMENTS 


•  CONTRACTORS  COMMENTS 

•  RELEASED  DRFP 

•  CONTRACTORS  COMMENTS 

•  RELEASED  FORMAL  RFP 

•  RECEIVED  EXTREMELY  COMPETITIVE  PROPOSALS 

•  SELECTED  THREE  VtflNNERS  FOR  STAGE  1 

•  DESIGN  REVIEWS 

•  DRAFT RFP STAGE  2 


JAN  76 
FEB  MAR  76 
MAR  76 
MAR  APR  76 

AUG  76* 
SEP  76 
DEC  76 
MAR  JUL-SEP  DEC  78 
SEP  79 


•  GOVERNMENTIINDUSTRY  PROCUREMENT  REVIEW  PANEL  DEC  79 

•  FORMAL  RFP  MAR  80 

•  3  MTH  DELAY  DUE  TO  HIGHER  HO  COORDINATION  PROBLEM 


GPS  CONTROL  SEGMENT 


1  «D/K  1 

(MN  79>M4Y  60) 


I  um-  ]  [CONTUOI.  SKGMKNT  -  STACK  2  IfINNKK 


<JWI  TO-MAY  60) 


(SRF  60  -  MAK  67) 


1  ™v  i 

(JAN  Tt-HAY  60) 


I  IJNIICON.  INC.  -  IVAV  (PNII  7t-JUN  66)  |  | 


CASE  STUDY  (1) 

•  PROGRAM  OFFICE  FORMS  SMALL  TEAM  O  S  PERSONNEL) 

TO  WORK  PENDING  ACQUISITION  SEP  77 

•  CBD  SYNOPOSIS  DEVELOPED  AND  ISSUED  SEP  77 

•  26  RESPONSES  RECEIVED  FROM  INDUSTRY  OCT  77 


•  13  CONTRACTORS  DEEMED  POTENTIAL  PRIMES  OCT  77 


COMPETITIVE  ACQUISITION  PLAN  CONTROL 
SEGMENT 


CASE  STUDY  (2) 


OrAQtl 
FNE066I0N 
17  MONTHS 


•  3  PARALLEL  CONTRACTS  •  COMPETITIVE  OC9ION 

•  FPPilOE)  •  COMMERCIAL  EQUIPMENT 

•  13  6M  EACH 


PROGRAM  OFFICE  CONDUCTED  CLOSED  BUSINESS  STRATEGY  NOV  77 
FOR  13  CANDIDATE  PRIMES 


11A&&1L 

FULL  SCALE  ENOINEER'NO/  •  SINGLE  CONTRACTOR 
DEPLOYMENT  •  FPIF/AF 

•  SHARE  LINE  76/30 

•  CCIIINO  130% 

•  (INCLUDINQAlLOPTlONtANO 
AWARD  PEE)  tIOM 


•  FULLY  OPERATIONAL 
CONTROL  SYSTEM 

•  UPORADBANOOSMOFfl 
SYSTEM 


•  PROGRAM  OFFICE  REQUESTED  PRIMES  TO  COME  BACK  IN  NOV  77 
TEN  DAYS  AND  GIVE  THEIR  CORPORATE  VIEW  OF  HOW 

BEST  TO  CONDUCT  ACQUISITION 

•  NINECONTRACTORSRESPONOEOWITHBRIEFtNUb  DEC  77 
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CASE  STUDY 

CASE  STUDY  (3)  BENEFITS  TO  MISSION  ASSURANCE 


•  NONE 

•  PROORAM  OFFICE  OEVELOPEO  BUSINESS  STRATEGY 

BASEO  ON  CONTRACTOR  INPUTS  DEC  TT 

•  LITTLE 

•  STRATEGY  BRIEFED  WITHIN  QOVERNMENT  DEC  TT 

•  MUCH 

•  CONDUCTED  OPEN  FORUM  ON  PROGRAM  WITH  ALL  INTERESTED 

INDUSTRY  PARTICIPANTS  JAN  71 


•  ALSO  RELEASED  TOP  LEVEL  GOVERNMENT  REQUIREMENTS  JAN  T« 

_  SUMMARY  (1) 


CASE  STUDY  (4) 


•  RECEIVED  CONTRACTOR  COMMENTS  FEB  A  MAR  Ti 

.  FORMAL 
•  INFORMAL 

•  RELEASED  DRAFT  RFP  MARTS 

•  RECEIVED  COMMENTS  APRTE 

•  RELEASED  FORMAL  RFP  AUG  TE  * 

•  THREE  CONTRACTORS  SELECTED  FOR  STAGE  I  DECTS 


‘  3  MONTH  DELAY  DUE  TO  HIGHER  HEADQUARTERS  COORDINATION 
PROBLEM 


PROCESS 


TOO  SHORT 
TOO  LONG 


EXPENSE 


TOO  LITTLE 
TOO  MUCH 


SUMMARY  (2) 


REQUIREMENTS 


TOO  LOOSE 
TOO  DETAILED 


CASE  STUDY  (5) 


TECHNICAL  - -  TOO  MUCH  LEVELINQ 

APPROACH  "  ■  TOO  LITTLE  LEVELINQ 


•  3  CONTRACTORS  IN  COMPETITION  FOR  IS  MONTH 

DESIGN  EFFORT  JAN  TS 

•  CONTINUOUS  IN  TERACTION  WITH  CONTRACTORS  BOTH  BUSINESS 

SIDE  A  TECHNICAL  SIDE  JAN-DEC  TS 

•  OOVERNMENTIINDUSTRY  PROCUREMENT  REVIEW  PANEL  DEC  TS 

•  FORMAL  STAGE  3  RFP  RELEASE  MAR  SO 


SUGGESTED  IMPROVEMENTS 

A 

B 

C 


•  EXPECT  ANNOUNCFMENT  OF  WINNER 


JULSO 


0 


COMMERCE  BUSINESS  DAILY 


SYNOPSES  OF  PROPOSED  ACQUISITIONS 


WHAT  IS  IT? 


A  DAILY  LIST  OF  U  S  QOV'T  ACQUISITION  INVITATIONSfREQUESTS. 
CONTRACT  AWARDS.  SUBCONTRACTINO  LEADS.  SALES  OF  SURPLUS 
PROPERTY.  AND  FOREIQN  BUSINESS  OPPORTUNITIES. 


GENERAL  POLICY 

(OAR  1-1001) 


"IT  IS  000  POLICY  TO  INCREASE  COMPETITION  BY  PUBLICIZINO 
ACQUISITIONS  WHICH  OFFER  COMPETITIVE  OPPORTUNITIES  FOR 
PROSPECTIVE  PRIME  CONTRACTORS  OR  SUBCONTRACTORS.  THUS 
ASSISTING  SMALL  BUSINESS  AND  LABOR  SURPLUS  AREA  CONCERNS  AND 
BROADENINO  INDUSTRY  PARTICIPATION  IN  DEFENSE  ACQUISITION 
PROORAMS." 


RFP’S/IFB’S 


CLEARLY  INDICATE  ANY  QUALIFYING  FACTORS  AFFECTING  THE 
ACQUISITION 


•  AVAILABILITY  OF  SPECIFICATIONS.  PLANS,  OR  DRAWINGS 

•  COMPLETE  DATA  NOT  AVAILABLE 

•  SECURITY  REQUIREMENTS 

•  AVAILABILITY  OF  BACKGROUND  RESEARCH  REPORT 

•  PRODUCTION  REQUIREMENTS 

•  STANDARDIZATION  REQUIREMENTS 


SYNOPSES 


“DESCRIPTION  SHALL  BE  CLEAR.  CONCISE.  AND  IN  SUCH  DETAIL  THAT 
IT  WILL  BE  UNDERSTOOD  BY  POTENTIALLY  INTERESTED  PARTIES.” 


•  ACOUISIIIUNSINEXCESSOFIIO.OOO 


*  MODS  TO  FXISTINO  CONTRACTS  WHEN  NEW  FUNDS  ARE  OBLIGAIED 


•  PRE  INVHATIONfSOLICITATION  NOTICES 


•  AOVANCr  NOnCE  OF  AN  AGENCY'S  INTEREST  IN  A  GIVEN  FIELD  or 
RAD 


•  SOUMCI  S  SOUQHT.  QOV'T  INTEREST  IN  SPECIFIC  RAD  TASK 


SYNOPSES  IN  CBD 

3  TYPES 


•  OF  PROPOSED  ACQUISITIONS  (DAR1-1003) 

•  OF  SUBCONTRACT  OPPORTUNITIES  (DAR  1-1003.6) 


•  OF  CONTRACT  AWARDS  (DAR  1  1005.1, 2-409) 


ADVANCE  NOTICE  OF  AN  AGENCY’S 
INTEREST 


A  LEAS|/BUY  ANALYSIS  FOR  STRATEGIC  FORCES  COMMUNICATION 

S(f^r^tTj  (srnAi  SA  r)  program,  this  notice  is  issued  to  obtain 

INDUSTRY’S  COMMENTS  OF  THE  POTENTIAL  FOR  THE  USAF  LEASING  THE 
STRATSAT  pnOQIIAM  THE  USAF  STRATSAT  PROGRAM  OFFICE  HAS 
DEVELOPED  A  SET  OF  ASSUMPTIONS  AND  POSSIBLE  HYBRID  LEASING 
CONCEPTS  THAT  WILL  BE  USED  IN  THE  ANALYSIS.  INTERESTED  FIRMS  MAY 
REQUEST  A  COPY  Ol  THE  ASSUMPTIONS  AND  CONCEPTS  FOR  COMMENT 
ON  THE  REALISM  OF  THE  PROGRAM  FROM  SPACE  DIVISION  NOT  LATER 
THAN  15  DAYS  AFTER  PUBLICATION  OF  THIS  NOTICE.  SEE  NOTE  99. 


PROPOSED  ACQUISITION 

POSSIBLE  SUBCONTRACTING  OPPORTUNITIES 


A  NAVSTAROIOIIAI  I'OSITIONINQ  SYSTFM  (OPS)  CONJROJLLABLE 
RECWIONPATUMN  ANTENNA (CRPA) FLIGHT  TEST  PROGRAM.  THE  JOINT 
PROGRAM  OFFICL  I  OR  THE  NAVSTAR  OPS  INTENDS  TO  CONDUCT 
NEGOTIATIONS  WITH  AIL  DIVISION.  CUTLER  HAMMER  ORQUP.  EATON 
CORPORATION.  COMAC  ROAD.  DEER  PARK.  LI,  NY  1 1 730  ON  OR  ABOUT 
17 SEPTEMBER  711  SAM50  UP  79-17  APPLIES  THE  CONTRACTOR  WILL  BE 
ASKED  TO  SUPPORI  A  C  141  CRPA  FLIGHT  TEST  PROGRAM  OF  EQUIPMENT 
DELIVERED  TO  THF  GOVERNMENT  UNDER  AN  INDEPENDENT  AIL  IRAO 
EFFORT  THIS  SYNOPSIS  IS  FOR  INFORMATION  AND  PLANNING  PURPOSES. 
DOES  NOT  CONSrilUTF  AN  IFO  OR  RFP.  AND  IS  NOT  TO  BE  CONSTRUFO 
AS  A  COMMITMENT  l)Y  THE  GOVERNMENT. 


NOTICE  OF  CONTRACT  AWARD 


A  THE  OOVrnNMI  NT  MAS  AWARDED  A  CON1MACT  TO  ROCKWELL 
INTERNATIONAL  CORP  .  400  COLLINS  ROAD  NEW.  CEDAR  RAPIDS,  lA  (L) 

(FOAToi  7r(:  oiriwro4/oi79  c  ooe3)  the  contractor  will  provide  full 

SCALE  ENOINT  (  MINU  DEVELOPMENT  FOR  THE  GLOBAL  POSITIONING 
SYSTEM  PHASf  IfR  USER  EOUIPMENT  AT  A  PRICE  OF  SM.  108,714.00 


80 FREQUENCY  RECEI^  -  2  EA  TO  INCLUDE  EXTENDER  CAROS. 
test  cords  and  PECULIAR  PLUGS  REQUIRED  TO  ALIGN,  TEST.  AND 
MAINTAIN  THr  LQUIPMENT.  SOLID  STATE.  MODULAR  CONSTRUCTION. 

digital  nrAocur.  manual  tuning  with  capability  for  remote 

PROCESSOK  01-  MANUAL  CONTROL.  STANDARD  EIA  19  IN.  RACK 
MOUNTING  lir>/k30  VAC  t  10%  FROM  47  TO  09  Hi.  FREQUENCY  RANGE 
S  TO  90  MHz;  MOOES.  CW.  AM.  LS8.  AND  USB.  ESTIMATED  RELEASE  DATE 
OF  IFB  F04703  70  B  0005  IS  17  AUGUST  1970.  THE  AIR  FORCE  RESERVES  THE 
RIGHT  TO  SET  THIS  PROCUREMENT  ASIDE  FOR  SMALL  BUSINESS  BASED  ON 
THE  RESPONSES  TO  THIS  SYNOPSIS.  SEE  NOTES  92.  60.  AND  99. 


SPECIFYING  THE  REQUIREMENTS  (CONT.) 


DRAFT  SOW/DRFP  LFFFCTIVFNtSS 
Mr.  Dick  Pooch,  LMSC  SSD 


DRAFT  SOW/RFP  EFFECTIVENESS 

•  SPECIFYING  THE  REQUIREMENTS 

•  STRUCTURING  THE  SOW 


•  IXlrUIIVI  SUMMARY  IIIGHIIOIIIS  ()HM(.|IVI  S  I'RIORIIIIS 
CONf  I  RNS  AND  (  ONSIRAINIS 

•  I'HOVIDI  KAIIONAIM 'JR  Kl  OUIKI  MINIS 

<.OVI  HNUI  Nl  '.until  *.  ANAI  V'.l  ANIt  II  '.I'. 


•  MOTIVATING  THE  CONTRACTORS 

•  MAXIMIZING  MISSION  ASSURANCE  BENEFITS 


SPECIFYING  THE  REQUIREMENTS 

•  MISSION  OlFINIIION 


01  SIGN  SOlUIIONS 


•  IRADFABlt  RFQUIRFMFNIS 

I’lKiOHMANI  I 
%(  Ml  1)1111 
(  OS  I 
RISK 

•  MUSIS  VS  WANTS 


SPECIFYING  THE  REQUIREMENTS  (CONT.) 

•  /IRO  BASI  SRKIMCAIIONS 

WHO  SI'I  <  II  II  S) 

I  ON  IHAI  I  OH  IINIOIII  SI'I  I  II  II  A  HONS 

•  SRICIflCAIION  lAllORING 

I'HI  VS  I'OSI  HI  I'  iSollHI  1  I  VAI IIAIION  I’HOMl  I  MSI 
(  OSI  AVI  )IIIANI  I  I'Oll  N  HAI 

•  SPICIFICAMON  I’VRAMIDS 

I'HODIII  I  ,  I  II  VI  IS  MAHIMIIM) 

Mil  SI'I  IS  SIANDAHIIS  III' OI  III!  II  I  III  HI,! 

•  CONITNI  DISCIPlINT 

Sll  HON  I  III  IINII  Al  ONIY 

AVOID  MANAOIMI  Nl  I'HOI  IIHI  Ml  N I  SOW  (  DKI  HI  DHIKI  Ml  N IS 
BUHIII)  IN  II I  IINII  Al  SI'I  I  II  11  A I  IONS 


STRUCTURING  THE  SOW 

•  OBJECTIVES  VS  TASKS 

•  WHAT  VS  HOW 

•  LEVEL  OF  DETAIL 

•  GOVERNMENT  VS  CONTRACTOR  GENERATION 

•  WBS  TRACEABILITY 

•  MIL  SPECS/STANDARDS/CDRLS 


MOTIVATING  THE  CONTRACTORS 


•  INDUSTRY  RtVIfW  COMMENT  CYCLE 
IIIHNAHOlIND  SI  III  null 
(.HOUND  nun  s  iohmai  (OnMhainis 
WHIIIIN  vs  OHAl 

OOVIHNMINI  INDUSIHY  MlIHUI  I^^AHI^I 

- 
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MOTIVATING  THE  CONTRACTORS 


MAXIMIZING  MISSION  ASSURANCE 
BENEFITS  (CONT) 


•  I’ROI’RIUARY  DAIA  PROItCIION 

DISMMINAIION  (>l  rOMMINIS  Al  COMMI  NDAMONS 
INII)KI'OHAIING(()MMINISIN  IHl  RFP 

•  COSIS  lO  INDUSIRY 


•  MAXtMIZl  OI’IN  GOVIRNMI  Nl  CONIRACIOR 
INURCHANGl  IHROUGHOUl  PROCESS 

•  PROIECI  PROPRIIIARY  INfORMAKON 

•  PROVIDE  VISIBILIIY  AND  FEEDBACK  OF  UEILI2AIION 
OF  CONIRACIOR  INE’UI 


MOTIVATING  THE  CONTRACTORS  (CONT.) 

•  INCENTIVIZING  CONTRACTORS 

EARLY  DEIAIIED  EVAIUAIION  CRIIERIA 

EVALUAIION  POINE  BONUS 

FAVORABLE  INIERCHANGE  ENVIRONMENT 

COSI  REIMBURSEMENI  CONSIDERATIONS 

ACCOMMODATION  FOR  INNOVATIONS 
(AVOIDING  alternate  PROPOSALS) 

GOVERNMENI  FEEDBACK 

EVIDENCE  OF  GOVERNMENT  ACCEPTANCE 


MAXIMIZING  MISSION  ASSURANCE 
BENEFITS 

•  EARLY  REQUIREMENTS  DISSEMINATION 


•  MAXIMIZE  CONTRACTOR  DEFINITION  RESPONSIBILITY 
UNIQUE  SPECS 


•  ENCOURAGE  PERFORMANCE  COST  RISK  REQUIREMENTS 
AND  CONCEPT  TRADES  ' 


DKAl'"!'  SOW/DUl'l'  Kl''l''W';i' 1  Vl'.Nl';SS 

Robert  J.  Crossley 
Procuring  Contracting  Officer 
Air  Force  Space  Division 

1.  My  presentation  is  based  on  my  exper¬ 
ience  at  Space  Division  and  does  not 
necessarily  reflect  the  procedures  used 
at  any  other  Government  buying  office. 

As  a  matter  of  fact  these  procedures  are 
not  always  used  at  Space  Division;  they 
vary  depending  on  value,  source  selec¬ 
tion/selected  source,  and  urgency  of  the 
acquisition.  While  we  do  have  a  rather 
structured  procedure  we  also  have  a 
degree  of  flexibility,  so  that  all  pro¬ 
curement  situations  can  be  accoimiodated. 
It  would  take  forever  to  discuss  all  of 
the  "what  if"  procurement  situations, 
so  I'm  presenting  the  framework  of  our 
review  procedures  for  a  typical  competi¬ 
tive  acquisition  valued  at  over  $10M 
with  particular  emphasis  on  the  draft 
RFP  considerations. 

2.  Let  me  give  you  a  one  or  two  line 
description  of  each  review  and  return  to 
the  draft  RFP. 

3.  Requirement  Review 

0  PMD  -  General  direction  to  a  buying 
command  telling  it  to  go  accomplish  a 
broad  objective. 

0  Acquisition  Plan  -  prepared  by  the 
buying  command  by  way  of  a  reply  to  the 
PMD;  states  how  the  objective  is  to  be 
attained. 

0  SOW  -  Statement  of  the  Government's 
need;  may  be  general  or  specific  depend¬ 
ing  on  the  phase  of  the  ac(|uisi tion. 

0  G  In-House  SOW  Review  -  A  review  l»y 
functional  specialist  to  assure  appli¬ 
cable  directives  have  been  considered. 

0  DRFP  -  May  be  a  complete  or  partial 
version  of  the  final  RFP.  Intended  to 
be  a  means  of  having  industry  partici¬ 
pate  in  defining  Government  requirements. 

0  G  In-House  Review  -  Consolidates 
industry  comments  into  a  final  RFP, 
includes  a  source  selection  board  review 


0  G  Murder  Board  -  Chaired  by  senior 
procurement  staff  personnel.  Membership 
includes  legal,  administrative,  financial 
and  functional  specialists.  This  is  a 
page  by  page  review  of  the  entire  RFP. 

0  RFP  Changes-  Usually  issued  prior  to 
receipt  of  proposals  but  may  be  after. 

4.  Now  then,  returning  to  the  draft  RFP 
procedures.  A  draft  must  be  sent  out  on 
virtually  all  acquisitions  whether  they 
are  sole  sourced  or  competed.  Prior  to 
its  release  the  draft  undergoes  an 
abbreviated  procurement  staff  review  to 
assure  that  it  is  at  least  generally 
compliant  with  existing  directives. 
Without  this  review  the  draft  and  final 
might  appear  to  bo  for  completely  differ¬ 
ent  requirements.  With  the  review  there 
is  some  assurance  that  all  the  "ilitios", 
human,  safety,  and  other  factors  appear 
in  the  draft  as  they  are  proposed  to 
appear  in  the  final. 

5.  Draft  RFPs  are  then  sent  to  all  firms 
on  the  source  list;  the  source  list 
being  made  up  of  firms  that  responded  to 
a  "sources  sought"  synopsis. 

6.  Industry  comments  are  reviewed  by 
functional  specialists  within  the  Govern¬ 
ment  and  a  record  made  of  the  disposition 
of  each  comment  (accepted,  rejected, 
modified,  etc.).  Comim’nts  are  not  usually 
discussed  with  the  submitters  because  it 
would  take  an  awful  lot  of  time.  Al)Out 
the  only  thing  a  submitter  gets  is  a 
letter  acknowledging  receipt  of  the 
coiiiiients.  This  minimal  feedback  is  a 
"soft  spot"  in  the  DRFP  procedure  since 
significant  comments  could  he  rejected 
because  of  the  way  they  are  written. 
However,  the  sulmiitter  may  get  three 
more  chances  to  present  the  coimient;  at 

a  pre-sol iciati on  conference,  a  pre¬ 
proposal  conference,  and  at  discussions 
prior  to  submitting  a  best  and  final 
offer.  Now,  if  a  submitter  really  wants 
to  crusade,  the  coiiiiient  could  bo  suli- 
mitted  as  part  of  an  alternate  proposal. 

7.  Another  "soft  spot"  in  the  DRFP 
procedure  is  the  need  for  a  finii  to 
stradle  a  line  betwc'en  trying  to  influ¬ 
ence  a  solicitation  by  submitting  DRIP 
coiniH’nts  and  compromising  its  compc'ti  Live 
position.  Whatever  incc'til.ive  t.iu're  is  to 


submit  comments  must  be  self-generated 
by  the  firm  itself,  since  the  Government 
doesn't  offer  much.  As  a  matter  of  fact, 
the  Government  has  its  own  version  of 
the  contractor's  dilemma;  specifically, 
the  Government  has  a  much  larger  stake 
in  preserving  the  competition  then  it 
does  in  receiving  DRFP  comments,  yet 
comments  are  solicited. 

8.  Ideally  DRFP  comments  represent  a 
meaningful  compromise  with  Industry  and 
the  Government  keeping  competition  as 
an  overriding  concern. 

9.  I'm  not  in  a  position  to  comment  on 
the  overall  effectiveness  of  the  Govern¬ 
ment's  DRFP  initiatives,  however  on  the 
programs  I've  been  associated  with,  the 
DRFP  effort  has  been  worthwhile.  As  a 
minimum,  it  is  a  communications  medium 
that  allows  firms  to  derive  an  early 
bid/no  bid  position.  Some  coiiinents  are 
incorporated  into  the  final  RFP,  these 
include  contract  line  item  structuring 
to  facilitate  payments;  Government 
property  requirements,  CDRL  and  specifi¬ 
cation  tailoring.  One  of  the  larger 
benefits  is  the  identification  of 
ambiguities  in  the  solicitation  that  can 
be  cleared  up  in  the  final. 

10.  To  the  extent  the  DRFP  procedures 
contribute  to  Government  requirements 
being  clearly  conmunicated  to  industry, 
they  are  worthwhile;  they  lose  value  to 
the  extent  they  compromise  competition 
or  lengthen  the  acquisition  cycle.  I 
don't  know  of  any  situation  where  these 
negative  concerns  have  materialized  so 
in  my  view  the  DRFP  Initiatives  have 
producing  positive  results.  Even  these 
results  could  be  Improved  if  the  Govern¬ 
ment  could  stimulate  greater  Industry 
participation.  In  my  view  formulating 
the  stimulant  will  prove  to  be  a  noble 
endeavor  because  of  the  subjective 
judgments  that  would  have  to  be  docu¬ 
mented  almost  to  the  point  of  demonstra¬ 
ting  that  there  was  no  subjectivity 
involved  and  the  very  limited  one-on- 
one  dialogue  that  is  permitted  during  a 
source  selection. 

11.  Leaving  the  DRFP,  I'd  like  to  give 
you  a  few  words  on  how  the  Government 
structures  their  Statement  of  Work. 


12.  Strucutre  of  SOW 

0  State  the  Objective  -  The  degree  to 
which  requirements  are  defined  depends 
on  the  stage  of  the  acquisition.  For 
the  conceptual  stage  relatively  broad 
objectives  are  defined;  for  the  produc¬ 
tion  stage  the  objectives  are  rather 
explicit. 

0  Government  Vs  Contractor  Generated  - 
The  practice  of  incorporating  a  contrac¬ 
tor's  proposal  or  other  document  by 
reference  into  the  RFP  or  the  contract 
as  the  SOW  is  not  condoned  and  will  be 
discontinued. 

0  Level  of  Detail  -  SOWs  often  have 
to  be  read  and  Interpreted  by  persons 
of  varied  backgrounds  such  as  lawyers, 
buyers,  engineers,  cost  estimators, 
accountant,  and  functional  specialists 
in  production,  transportation,  security, 
audit,  quality  control  and  contract 
managers . 

0  Traceability  to  WBS  -  Although 
desirable  it  depends  on  the  particular 
program.  Multi-service  participation 
may  influence  WBS. 

0  What  To  Do  Vs  How  To  Do  It  -  Supply 
acquisitions  demand  definable  physical 
or  performance  descriptions  of  end 
products.  R&D  acquisitions  should  be 
definitive  enough  to  protect  the 
Government's  interest,  yet  broad  enough 
to  allow  for  the  contractor's  creative 
effort. 

0  Inclusion  of  CDRLs  and  Reporting  - 
Contract  data  or  reporting  requirements 
should  not  be  duplicated  in  the  SOW. 

DO  1423  is  the  medium  for  establishing 
data  requirements. 

13.  Thank  you. 
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SPACE  DIVISION  RFP  PREPARATION 
ROBERT  G.  COOK 

DSCS  II  Systems  Effectiveness  Manager 

This  presentation  is  intended  to  show  the 
Mission  Assureince  requirements  normally 
included  in  Air  Force  Space  Division  Re¬ 
quest  for  Proposals.  These  requirements 
are  differentiated  from  the  technical 
performance  requirements  and  are  loosely 
grouped  under  the  term  System  Effective¬ 
ness.  I  intend  to  show  the  Space  Divi¬ 
sion  philosophy  toward  the  Request  for 
Proposal,  outline  the  requirements  and 
directives  which  govern  preparation  of 
the  RFP  and  show  the  Internal  processing 
drafts  of  the  RFP  receive  before  they  are 
formally  issued.  Then  I  will  discuss  the 
outline  of  the  RFP  and  within  that  out¬ 
line  show  where  the  principal  System  Ef¬ 
fectiveness  requirements  are  located. 
These  requirements  are  not  really  hidden 
but  frequently  there  is  a  tendency  to 
only  look  for  requirements  within  the 
statement  of  work  or  the  compliance  docu¬ 
ments.  Lastly,  I  want  to  show  what  the 
source  selection  evaluation  team  is  look¬ 
ing  for  in  the  proposal  itself  and  point 
to  some  problems  we  have  experienced  in 
the  RFP-Proposal  cycle  and  request  sug¬ 
gestions  to  reduce  their  impact. 

To  begin  we  must  define  what  is  generally 
meant  by  system  effectiveness.  The  con¬ 
tract  requirements  shown  on  this  slide 
(3)  are  the  principal  tasks  normally 
grouped  under  the  systems  effectiveness 
label.  Some  people  add  and  some  subtract 
from  this  list  but  generally  it  covers 
the  ground.  These  are  the  requirements 
which  we  will  attempt  to  locate  in  the 
RFP. 

The  government  objectives  for  an  RFP  are 
shown  on  this  slide  (4) .  It  is  important 
to  recognize  that  the  RFP  is  a  means  to 
accomplish  an  end,  not  an  end  in  itself. 
Contrary  to  popular  belief,  it's  objec¬ 
tive  is  not  to  'trap'  a  contractor  into 
financing  the  United  States  Government 
either.  It  is  supposed  to  be  a  simple 
statement  of  the  government's  needs  and 
directions  on  how  to  bid  in  an  acceptable 
manner  to  supply  those  supplies  and  ser¬ 
vices  to  the  government.  The  primary 


purpose  is  the  last  item  on  the  chart 
"meaningful  negotiations".  Contracts 
require  a  mutual  understanding  of  the 
requirements  and  the  methods  expected 
to  be  applied  in  achieving  requirements. 
Negotiations  is  the  work  expended  in 
mutually  understanding  the  scope  of  work 
to  be  accomplished.  Effort  spent  during 
this  stage  of  the  program  avoids  almost 
continual  confrontations  over  interpre¬ 
tations  of  requirements  later  in  the 
progrcun . 

Space  Division  policy  since  1977  has  em¬ 
phasized  the  importance  of  mission  suc¬ 
cess.  Evaluation  factors  in  the  past, 
while  requiring  evaluation  of  quality, 
did  not  require  formal  mention  of  quality 
in  the  briefing  to  the  source  selection 
authority.  Now  the  perception  of  quality 
is  a  vital  factor  in  source  selection  and 
it  must  be  discussed  in  every  source 
selection  briefing.  This  is  tacit  recog- 
nizition  that  simply  building  a  space 
system  which  works  is  no  longer  a  major 
problem,  the  significant  problem  is  to 
build  a  system  which  works  over  a  long 
term  and  provides  continuous  service. 

The  directives  on  this  slide  (6)  are  the 
primary  source  documents  for  planning 
and  structuring  the  request  for  proposal. 
The  preparation  guide  not  only  lays  out 
the  required  content  of  the  RFP :  it  pro¬ 
vides  an  excellent  outline  of  the  Propo¬ 
sal  instructions  including  a  suggested 
table  of  contents .  The  pamphlets  on 
statement  of  work  preparation  are  also 
useful  guides .  Since  the  overall  purpose 
of  the  RFP  is  source  selection,  I  also 
show  those  requirements  here  because  any 
RFP  must  require  the  information  neces¬ 
sary  to  make  a  source  selection  decision. 

Chart  7  is  a  very  abbreviated  flow  chart 
of  the  RFP  cycle.  The  background  infor¬ 
mation  is  digested  and  separated  into 
contractor  and  government  activities. 

When  the  contractor  work  is  determined, 
the  RFP  generation  cycle  usually  starts 
with  a  Contracting  officer  and  a  Project 
officer  drafting  the  required  documenta¬ 
tion,  one  of  which  is  the  Request  for 
Proposal.  A  tremendous  cunount  of  coordi¬ 
nation  is  required  during  the  RFP  draft¬ 
ing  effort  as  all  interested  parties  pro¬ 
vide  input  information  and  comments  for 
incorporation.  It  is  really  up  to  the 
project  officer  to  assure  that  these  com- 


ments  fit  the  RFP  and  form  a  unified  and 
understandable  document.  After  prepara¬ 
tion  of  the  initial  drafts,  coordination 
with  industry  may  be  solicited  and  re¬ 
view  by  a  number  of  panels  occurs.  These 
reviews  are  usually  very  detailed  and 
agreement  must  be  reached  before  release 
of  the  RFP  is  allowed.  This  cycle  is 
extremely  long  and  time  consuming  and  in 
many  cases  the  panels  provide  conflicting 
advice  which  must  be  resolved.  The  cur¬ 
rent  trend  in  procurement  appears  to  be 
to  add  more  review  cycles  to  this  list, 
not  less. 

An  outline  of  the  RFP  is  shown  in  the 
next  slide.  I  will  discuss  each  of  these 
parts  and  show  their  system  effectiveness 
requirements.  This  outline  was  changed 
recently.  The  General  Instructions  sec¬ 
tion,  Part  IV,  was  originally  Part  I. 

The  current  Part  I,  II,  and  III  consti¬ 
tute  the  model  contract  portion  of  the 
RFP.  This  section,  with  all  the  blanks 
completed  eventually  becomes  the  contract 
and  should  contain  all  the  government  re¬ 
quirements.  The  contractors  proposal  in 
the  form  of  plans  is  usually  added  to 
part  four  to  commit  the  contractor  to  do 
the  work  he  proposed  and  on  which  he 
based  his  costs. 

Where  are  the  requirements?  They  are  well 
distributed  throughout  the  RFP.  The 
Schedule  documents  the  items  shown.  Its 
principal  feature  is  the  identification 
of  deliverable  products  and  services  ^md 
definition  of  how  and  when  the  government 
intends  to  verify  that  the  product  or  ser¬ 
vice  conforms  to  the  requirement.  The 
General  Provisions  of  the  contract  con¬ 
tain  those  standard  DAR  clauses  which 
apply.  There  is  occasionally  some  dupli¬ 
cation  of  these  clauses  with  the  state¬ 
ment  of  work  -  notably  Mil-Q-9858  is  nor¬ 
mally  cited  as  a  requirement  in  both. 

The  main  system  effectiveness  require¬ 
ments  are  in  part  III.  Here  the  State¬ 
ment  of  Work  tasks  are  referenced  includ¬ 
ing  the  compliance  documents.  Deliver¬ 
able  data  is  identified  and  described  in 
detail  and  the  specification  of  require¬ 
ments  is  referenced.  Mil-Std-454  speci¬ 
fication  requirements  for  space  vehicles 
is  tailored  by  Mil-E-8983B.  The  compli¬ 
ance  doctiment  list  of  the  statement  of 
work  is  where  most  management  system  re¬ 


quirements  are  invoked  Space  Division 
directives  are  very  clear  about  the  need 
for  tailoring  of  compliance  documents, 
and  only  using  those  portions  which  are 
necessary  for  the  procurement.  Tailoring 
should  be  documented  along  with  the  docu¬ 
ment  call-out  on  the  list.  It  is  usually 
not  sufficient  or  acceptable  to  tailor 
requirement  documents  in  the  contractors 
proposal  or  plans.  Contractor  input  to 
the  tailoring  process  is  desireable  and 
usually  solicited  by  issuing  "draft"  RFPs 
for  industry  coordination.  The  chart 
(12)  lists  some  of  the  compliance  docu¬ 
ments  which  normally  appear  on  SD  Requests 
for  Proposals. 

The  General  Instructions  portion  of  the 
RFP  contains  detailed  directions  for  prep¬ 
aration  of  the  Proposal.  It  will  usually 
contain  an  outline  of  the  Proposal  volume 
by  volume.  Each  section  identified  will 
include  a  description  of  the  data  to  be 
provided  and  analysis  results  to  be  pre¬ 
sented..  Plans  necessary  for  the  evalua¬ 
tion  of  the  proposal  will  be  identified. 
The  last  section  of  the  General  Instruc¬ 
tions  contains  the  evaluation  items  con¬ 
sidered  by  the  source  selection  evalua¬ 
tion  team  and  the  grading  criteria  and 
relative  importance  of  each.  Usually  the 
source  selection  team  is  organized  in  the 
manner  outlined  in  this  section  and  ap¬ 
plicable  portions  of  the  proposal  separ¬ 
ated  and  distributed  according  to  this 
overall  organization. 

Since  a  proposal  is  the  intended  end  re¬ 
sult  of  the  RFP  it  is  worthwhile  to  con¬ 
sider  what  response  the  government  expects 
to  it’s  RFP. 

First,  of  course,  the  government  expects 
the  contractor  to  identify  his  approach 
in  sufficient  detail  so  that  the  risks 
involved  in  the  approach  can  be  fully 
evaluated . 

Second,  the  government  expects  the  propo¬ 
sal  to  be  sufficiently  complete  to  deter¬ 
mine  the  work  tasks  to  be  performed  by 
the  contractor .  The  government  needs  this 
data  to  understand  the  scope  of  the  work 
proposed  and  the  cost  basis  of  the  con¬ 
tractor's  proposal.  It  also  supports  re¬ 
alistic  estimates  of  contract  schedule 
and  the  related  risks. 

The  proposal  is  required  to  contain  de- 
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SPACE  DIVISION  RFP  PREPARATION 


ROBERT  G.  COOK 

DSCS  II  Systems  Effectiveness  Manager 

This  presentation  is  intended  to  show  the 
Mission  Assurance  requirements  normally 
included  in  Air  Force  Space  Division  Re¬ 
quest  for  Prop>osals.  These  requirements 
are  differentiated  from  the  technical 
performance  requirements  and  are  loosely 
grouped  under  the  term  System  Effective¬ 
ness.  I  intend  to  show  the  Space  Divi¬ 
sion  philosophy  toward  the  Request  for 
Proposal,  outline  the  requirements  and 
directives  which  govern  preparation  of 
the  RFP  and  show  the  internal  processing 
drafts  of  the  RFP  receive  before  they  are 
formally  issued.  Then  I  will  discuss  the 
outline  of  the  RFP  and  within  that  out¬ 
line  show  where  the  principal  System  Ef¬ 
fectiveness  requirements  are  located. 

These  requirements  are  not  really  hidden 
but  frequently  there  is  a  tendency  to 
only  look  for  requirements  within  the 
statement  of  work  or  the  compliance  docu¬ 
ments.  Lastly,  I  want  to  show  what  the 
source  selection  evaluation  team  is  look¬ 
ing  for  in  the  proposal  itself  and  point 
to  some  problems  we  have  experienced  in 
the  RFP-Proposal  cycle  and  request  sug¬ 
gestions  to  reduce  their  impact. 

To  begin  we  must  define  what  is  generally 
meant  by  system  effectiveness .  The  con¬ 
tract  requirements  shown  on  this  slide 
(3)  are  the  principal  tasks  normally 
grouped  under  the  systems  effectiveness 
label.  Some  people  add  and  some  subtract 
from  this  list  but  generally  it  covers 
the  ground.  These  are  the  requirements 
which  we  will  attempt  to  locate  in  the 
RFP. 

The  government  objectives  for  an  RFP  are 
shown  on  this  slide  (4) .  It  is  important 
to  recognize  that  the  RFP  is  a  means  to 
accomplish  an  end,  not  an  end  in  itself. 
Contrary  to  popular  belief,  it's  objec¬ 
tive  is  not  to  'trap'  a  contractor  into 
financing  the  United  States  Government 
either.  It  is  supposed  to  be  a  simple 
statement  of  the  government's  needs  and 
directions  on  how  to  bid  in  an  acceptable • 
manner  to  supply  those  supplies  and  ser¬ 
vices  to  the  government.  The  primary 


purpose  is  the  last  item  on  the  chart 
"meaningful  negotiations" .  Contracts 
require  a  mutual  understanding  of  the 
requirements  and  the  methods  expected 
to  be  applied  in  achieving  requirements. 
Negotiations  is  the  work  expended  in 
mutually  understanding  the  scope  of  work 
to  be  accomplished.  Effort  spent  during 
this  stage  of  the  program  avoids  almost 
continual  confrontations  over  interpre¬ 
tations  of  requirements  later  in  the 
program . 

Space  Division  policy  since  1977  has  em¬ 
phasized  the  importance  of  mission  suc¬ 
cess.  Evaluation  factors  in  the  past, 
while  requiring  evaluation  of  quality, 
did  not  require  formal  mention  of  quality 
in  the  briefing  to  the  source  selection 
authority.  Now  the  perception  of  quality 
is  a  vital  factor  in  source  selection  and 
it  must  be  discussed  in  every  source 
selection  briefing.  This  is  tacit  recog- 
nizition  that  simply  building  a  space 
system  which  works  is  no  longer  a  major 
problem,  the  significant  problem  is  to 
build  a  system  which  works  over  a  long 
term  and  provides  continuous  service. 

The  directives  on  this  slide  (6)  are  the 
primary  source  documents  for  planning 
and  structuring  the  request  for  proposal. 
The  preparation  guide  not  only  lays  out 
the  required  content  of  the  RFP:  it  pro¬ 
vides  an  excellent  outline  of  the  Propo¬ 
sal  instructions  including  a  suggested 
table  of  contents .  The  pamphlets  on 
statement  of  work  preparation  are  also 
useful  guides.  Since  the  overall  purpose 
of  the  RFP  is  source  selection,  I  also 
show  those  requirements  here  because  any 
RFP  must  require  the  information  neces¬ 
sary  to  make  a  source  selection  decision. 

Chart  7  is  a  very  abbreviated  flow  chart 
of  the  RFP  cycle .  The  background  infor¬ 
mation  is  digested  and  separated  into 
contractor  and  government  activities . 

When  the  contractor  work  is  determined, 
the  RFP  generation  cycle  usually  starts 
with  a  Contracting  officer  and  a  Project 
officer  drafting  the  required  documenta¬ 
tion,  one  of  which  is  the  Request  for 
Proposal.  A  tremendous  emount  of  coordi¬ 
nation  is  required  during  the  RFP  draft¬ 
ing  effort  as  all  interested  parties  pro¬ 
vide  input  information  and  comments  for 
incorporation.  It  is  really  up  to  the 
project  officer  to  assure  that  these  com- 


ments  fit  the  RFP  and  form  a  unified  and 
understandable  document.  After  prepara¬ 
tion  of  the  initial  drafts,  coordination 
with  industry  may  be  solicited  and  re¬ 
view  by  a  number  of  panels  occurs.  These 
reviews  are  usually  very  detailed  and 
agreement  must  be  reached  before  release 
of  the  RFP  is  allowed.  This  cycle  is 
extremely  long  and  time  consuming  and  in 
many  cases  the  panels  provide  conflicting 
advice  which  must  be  resolved.  The  cur¬ 
rent  trend  in  procurement  appears  to  be 
to  add  more  review  cycles  to  this  list, 
not  less. 

An  outline  of  the  RFP  is  shown  in  the 
next  slide.  I  will  discuss  each  of  these 
parts  and  show  their  system  effectiveness 
requirements.  This  outline  was  changed 
recently.  The  General  Instructions  sec¬ 
tion,  Part  IV,  was  originally  Part  I. 

The  current  Part  I,  II,  and  III  consti¬ 
tute  the  model  contract  portion  of  the 
RFP.  This  section,  with  all  the  blanks 
completed  eventually  becomes  the  contract 
and  should  contain  all  the  government  re¬ 
quirements.  The  contractors  proposal  in 
the  form  of  plans  is  usually  added  to 
part  four  to  commit  the  contractor  to  do 
the  work  he  proposed  and  on  which  he 
based  his  costs. 

Where  are  the  requirements?  They  are  well 
distributed  throughout  the  RFP.  The 
Schedule  documents  the  items  shown.  Its 
principal  feature  is  the  identification 
of  deliverable  products  and  services  and 
definition  of  how  and  when  the  government 
intends  to  verify  that  the  product  or  ser¬ 
vice  conforms  to  the  requirement.  The 
General  Provisions  of  the  contract  con¬ 
tain  those  standard  DAR  clauses  which 
apply.  There  is  occasionally  some  dupli¬ 
cation  of  these  clauses  with  the  state¬ 
ment  of  work  -  notably  Mil-Q-9858  is  nor¬ 
mally  cited  as  a  requirement  in  both. 

The  main  system  effectiveness  require¬ 
ments  are  in  part  III.  Here  the  State¬ 
ment  of  Work  tasks  are  referenced  includ¬ 
ing  the  compliance  documents.  Deliver¬ 
able  data  is  identified  and  described  in 
detail  and  the  specification  of  require¬ 
ments  is  referenced.  Mil-Std-454  speci¬ 
fication  requirements  for  space  vehicles 
is  tailored  by  Mil-E-8983B.  The  compli¬ 
ance  document  list  of  the  statement  of 
work  is  where  most  management  system  re¬ 


quirements  are  invoked  Space  Division 
directives  are  very  clear  about  the  need 
for  tailoring  of  compliance  documents, 
and  only  using  those  portions  which  are 
necessary  for  the  procurement.  Tailoring 
should  be  documented  along  with  the  docu¬ 
ment  call-out  on  the  list.  It  is  usually 
not  sufficient  or  acceptable  to  tailor 
requirement  documents  in  the  contractors 
proposal  or  plans.  Contractor  input  to 
the  tailoring  process  is  desireable  and 
usually  solicited  by  issuing  "draft"  RFPs 
for  industry  coordination.  The  chart 
(12)  lists  some  of  the  compliance  docu¬ 
ments  which  normally  appear  on  SD  Requests 
for  Proposals. 

The  General  Instructions  portion  of  the 
RFP  contains  detailed  directions  for  prep¬ 
aration  of  the  Proposal.  It  will  usually 
contain  an  outline  of  the  Proposal  volume 
by  volume.  Each  section  identified  will 
include  a  description  of  the  data  to  be 
provided  and  analysis  results  to  be  pre¬ 
sented.  Plans  necessary  for  the  evalua¬ 
tion  of  the  proposal  will  be  identified. 
The  last  section  of  the  General  Instruc¬ 
tions  contains  the  evaluation  items  con¬ 
sidered  by  the  source  selection  evalua¬ 
tion  team  and  the  grading  criteria  and 
relative  importance  of  each.  Usually  the 
source  selection  team  is  organized  in  the 
manner  outlined  in  this  section  and  ap¬ 
plicable  portions  of  the  proposal  separ¬ 
ated  and  distributed  according  to  this 
overall  organization. 

Since  a  proposal  is  the  intended  end  re¬ 
sult  of  the  RFP  it  is  worthwhile  to  con¬ 
sider  what  response  the  government  expects 
to  it's  RFP . 

First,  of  course,  the  government  expects 
the  contractor  to  identify  his  approach 
in  sufficient  detail  so  that  the  risks 
involved  in  the  approach  can  be  fully 
evaluated. 

Second,  the  government  expects  the  propo¬ 
sal  to  be  sufficiently  complete  to  deter¬ 
mine  the  work  tasks  to  be  performed  by 
the  contractor.  The  government  needs  this 
data  to  understand  the  scope  of  the  work 
proposed  and  the  cost  basis  of  the  con¬ 
tractor's  proposal.  It  also  supports  re¬ 
alistic  estimates  of  contract  schedule 
and  the  related  risks. 

The  proposal  is  required  to  contain  de- 
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tailed  work  plans.  These  plans  are  not 
intended  to  be  a  rehash  of  the  compliance 
documents  but  detailed  work  plans  describ¬ 
ing  who  will  do  what  work,  when  that  work 
will  be  done,  and  the  methods  and  number 
of  people  intended  to  work  the  task.  It 
is  amazing  that  most  proposals  do  not  use 
this  simple  approach  to  planning.  If  this 
plan  goes  into  sufficient  detail  there 
will  be  no  question  about  the  intentions 
or  understanding  of  the  requirements  by 
the  contractor.  When  the  proposal  plans 
are  written  in  this  straightforward  fash¬ 
ion  the  negotiation  effort  is  reduced  be¬ 
cause  both  parties  will  understand  what 
work  is  to  be  done. 

The  last  chart  (16)  lists  some  examples 
of  perceived  problems  with  RFPs.  Most  of 
these  problems  will  never  go  away  com¬ 
pletely,  but  work  to  reduce  the  irrita¬ 
tion  level  would  certainly  be  useful. 


AGENDA 


WHERE  ARE  SYSTEM  EFFECTIVENESS  REQUIREMENTS  IN  RFP^ 
WHAT  RESPONSE  DOES  THE  GOVERNMENT  EXPECT? 

PROBLEMS  EXPERIENCED. 


SETTING  THE  STAGE 


SYSTEM  EFFECTIVENESS  IS  A  SYNTHESIS  OF  THE  CONTRACTING  "ILITIES" 

-  RELIABILITY  -  PRODUCIBILITY 

-  MAINTAINABILITY  -  OPERABILITY 

-  QUALITY 


REQUEST  FOR  PROPOSAL  OBJECTIVE 

•SOLICIT  PROPOSALS  FOR  REOUIRED  SUPPLIES  AND  SERVICES  FROM  INDUSTRY." 

■KEY  COUWNICATION  TO  POTENTIAL  CONTRACTORS  ON  EXACTLY  WHAT,  HOW,  AND 
WHEN  THE  GOVERNMENT  NEEDS  TO  BUY’ 

PROVIDE  A  VEHICLE  TO  ’FOSTER  MEANINGFUL  NEGOTIATIONS' 


SPACE  DIVISION  POLICY 

'WHILE  MANY  FACTORS  MUST  BE  CONSIDERED  DURING  THE  SELECTION 
PROCESS,  QUALITY  ASSURANCE  (QA)  MUST  BE  INCLUDED  SINCE  IT  IS 
MOST  VITAL  TO  MISSION  SUCCESS.  IT  IS  FiniNG,  THEREFORE,  THAT 
THE  EVALUATION  FACTORS  FOR  AWARD  IN  ALL  SOURCE  SELECTION  PLANS 
AND  COMPETITIVE  SOLICITATIONS  CLEARLY  SET  FORTH  THIS  DOMINANT 
REQUIREMENT  AS  AN  AWARD  CONSIDERATION.' 


DIRECTIVES  RELATED  TO  RFP  PREPARATION 


•  AFSC  PAMPHLET  70-A 

"REQUEST  FOR  PROPOSAL  PREPARATION  GUIDE" 

•  AF  REGULATION  70-15 

"SOURCE  SELECTION  POLICY  AND  PROCEDURES" 

I  AFSC  REGULATION  70-15 

"SOURCE  SELECTION  POLICY  AND  PROCEDURES" 

•  AFSC  PAMPHLET  800-6 

"STATEMENT  OF  WORK  PREPARATION  GUIDE" 

•  SPACE  DIVISION  PAMPHLET  800-6 

"STATEMENT  OF  WORK  PREPARATION  GUIDE" 


IM  IlfF  PlitfAIIAlim  CtCU 
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•  ETC. 
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CONTENT  OF  THE  RFP 


ilHERE  ARE  THE  REOUIREMENTS? 


I  PART  I  -  THE  SCHEnULE 

SUPPLIES,  services  AND  PRICES 
DESCRIPTION/SPECIFICATIONS 
PRESERVATION/PACKAGINfi/PACKING 
PELIVFRIES  ON  PERFORMANCE 
INSPECTION  AND  ACCEPTANCE 
SPECIAL  PROVISIONS 
CONTRACT  ADMINISTRATION  DATA 

•  PART  II  -  GENERAL  PROVISIONS 

GENERAl  PROVISIONS 

•  PART  II!  -  LIST  OF  fflCUMENTS  AND  AHACHMENTS 

STATEMENT  OF  WORK 
COMPLIANCF  DOCUMENTS 
CORL 

•  PART  IV  -  GENERAL  INSTRUCTIONS 

COVER  SHEET 

CONTRACT  FORMS,  REPRESENTATIONS  AND  CERTIFICATIONS 
SOLICITATIONS  INSTRUCTIONS  CONDITIONS  AND  NOTICE  TO  OFFERS 
EVALUATION  FACTORS 


WHERE  ARE  THE  REQUIREMENTS? 


PART  I  THE  SCHEDULE 


IDENTIFIES  ITEMS  TO  BE  DELIVERED 
INCLUDES  SITE  CODES  FOR  PQA/ACP/FOB. 


IDENTIFIES  PRICE  AND  DELIVERY  DATES. 


INSPECTION,  ACCEPTANCE  AND  FOB  POINT 

DEFINES  GOVERNMENT  PARTICIPATION  IN  INSPECTION  AND 
ACCEPTANCE  ACTIVITY, 

WHERE  ARE  THE  REniHREMENTS? 


PART  II  -  GENERAL  PROVISIONS 


-  STANOARO  OF  WORK  7-302.2 

-  INSPECTION  CLAUSES  7-302. 4(a) 

7-302.4(8) 

-  SERIAL  INSPECTION  »  RECEIVING  7-104.62 
REPORT 


-  QUALITY  PR0GRA1  7-104.28 


-  CERTIFICATE  OF  CONFORMANCE  7-104.100 


PART  III  LIST  OF  DOCUMENTS,  EXHIBITS  AND  OTHER  ATTACHMENTS 

STATEMENT  Of  WORK 

PARAGRAPH  3;  CONTRACTOR  TASKS 

-  3.1  COMPLIANCE  DOCUMENTS 

-  3.x  OTHER  TASKS 

PARAGRAPH  A;  SPECIAL  CONSIDERATIONS 

-  CDRL 

-  PLANS  A  UPDATES 

-  SPECIFICATIONS 

-  FAILURE/DEFICIENCY  REPORTS 

-  TEST  PLANS/PROCEDURES/REPORTS 

-  SYSTEM  SPECIFICATIONS 

-  SPECIFIC  HARDWARE  REOUIREMENTS  USUALLY  FROM  MIL-STD-RSA 


WHERE  ARE  THE  REQUIREMENTS? 

COMPLIANCE  DOCUMENTS  OF  SOW 

-  SAHSO  STD  n-2  ELECTRONIC  PARTS,  MATERIALS  AND  PROCESSES  FOR  SPACE 
AND  MISSILE  APPLICATIONS;STD  CONTROL  PROGRAM 

-  SAMSO  STD  73-5  SPECIAL  QUALITY  ASSURANCE  REOUIREMENTS  FOR  SPACECRAFT 
AND  MISSILE  SYSTEMS 

-  MIL-STD-1520  CORRECTIVE  ACTION  AND  DISPOSITION  SYSTEM  FOR  NON  CONFORMING 
MATERIAL 

•  HIL-STD-15A3 

-  MIl-i»-9858 

-  HIL-S-52779 
CONTRACTOR  PLANS 


WHERE  ARE  THE  REOUIREMENTS? 


PART  IV  GENERAL  INSTRUCTIONS 

BACKGROUND  OF  PROGRAM 

PROPOSAL  FORMAT 

WHAT  PUNS  ARE  REQUIRED? 

WHAT  ANALYSIS  IS  REQUIRED? 

HOW  COSTS  ARE  TO  BE  STATED 

-  EVALUATION  FACTORS 

IDENTIFIES  EVALUATION  BREAKOUT  AREAS,  ITEMS,  FACTORS. 

IDENTIFIES  CRITERIA  ADEOUACY/DEFINITIVENESS,  CURITY/COMPLETENESS. 
IDENTIFIES  REUTIVE  RANKING. 


WHAT  RESfONSF.  IS  IXt'ECFlD? 


PROPOSAL  WHICH  ADDRESSES  EACH  OF  THE  'ility'  REOIIIREHENTS 

IDENTIFY  AREAS  OF  TECHNICAL  RISK 
nETIiODS  TO  REDUCE  RISK 
1IMETABLL  FOR  DECISION 

PROPOSAL  SHOULD  CONTAIN  DETAILED  PLANS 

EXPLAIN  IIIF  MANAWnENT  OF  THE  lASK 
WHO  DOES  WHAT 

HOW  WORK  Will.  BL  DONE  (MriHOOS/EXlENT) 

WHEN  WORK  Will  HE  DONE 
-  SUIT-URILD  BY  CUSl  ANALYSIS 
DO  NOI  RESIAll.  RFOUIREMFNTS 


WHAT  RESIYINSL  IS  EXPECTED? 

PROPOSAL  WHICH  PROVIDES  A  MEANS  TOR  CONTRACT  DETINITI/ATION 

-  DEMONSTRATES  UNDERSTANDING  OF  REOUIREMFNTS 

-  DEFINES  WORK  CONTRACTOR  WILL  PERFORM 

-  SUPPORTS  A  ’FAIR  FXCHANriE' 


rMiiiiins  ixi'iiiliNdli 

ixiviiiwiirii  KiuiiiHinNis  I  mil  iimim  imnininiis.  (ni'iii;  anh  siawiariki 
nANAiiinNi  iHiiiiniiiis  hiari.i  .mi  iasi  iaiii  (imiiiAii  i::  'iiiw  iiaii  i.An'. 

UINIKAl  lim  I'HIIIHSAIS  IKI  NlllllINl.  f«im  IIIAN  HII'IAl  IIIUdlHIHIIIt';  IWW  Ml'. 
IHbl  I'WIIMSAIS  III  mil  SIII'IHIII  lAl.K'l  I'KIIHNII. 

IVAIIIAIIIIN  III  I'RnillSAlS  IS  mi'Almil  IIY  lAIIIIIII  III  inNSimK  IVAIIIAIIIM 
IN  llimillNI.. 


PREPROPCSAL  PAPER 
Jeremiah  J.  Madden 
PROJECT  MANAGER,  GRO,  GSFC 

1 .  Introduction 

This  paper  will  deal  with  the  contents  of 
Request  for  Proposals  (RFP)  for  Space 
Transportation  System  (STS)  payloads  from 
the  Goddard  Space  Flight  Center  (GSFC). 
The  STS  defines  a  payload  as  an  equip¬ 
ment  or  material  carried  by  the  STS  that 
is  not  considered  part  of  the  basic  STS 
itself,  including  items  such  at.  free- 
flying  automated  spacecraft,  coherent  ex¬ 
periment  units,  individual  experiments 
and  instruments.  In  general,  such  pay- 
loads  are  going  to  be  procured  using  a 
two  phase  approach  where  the  first  phase 
calls  for  a  study  proposal  and  the  second 
phase  calls  for  an  implementation  pro¬ 
posal.  In  the  study  phase,  one  or  more 
proposals  are  accepted.  An  RFP  for  a  one 
proposal  approach  would  be  similar  to  an 
RFP  issued  in  Phase-II  of  a  two  phase 
approach. 

In  general,  for  large  procurements,  NASA 
Management  Instruction  (NMI)  1700. 14A 
(Major  Ssytem  Acquisition)  will  be  used. 
Tins  is  NASA's  implementation  of  the 
I'ffice  of  Management  (0MB)  Circular  A-109. 
The  author  is  most  familiar  with  the 
Gamma  Ray  Observatory  (CRO)  procurement, 
and  in  most  Instances  will  be  basing  his 
remarks  on  this  particular  experience. 

.  Early  Briefings 

Normally  the  call  for  scientific  instru¬ 
ment  proposals  precedes  the  release  of  an 
RFP  for  a  prime  or  mission  contractor  by 
several  months  to  a  few  years.  The 
Announcement  of  Opportunity  (AO)  gives 
general  information  about  the  program's 
plans  and  objectives.  An  AO  gives 
general  orbit  and  spacecraft  character¬ 
istics  as  well  as  a  description  of  the 
evaluation,  selection  and  acquisition 
processes.  It  alert.s  industry  to  NASA's 
intent  to  proceed  with  the  program. 

When  possible,  NASA  will  brief  industry 
on  its  upcoming  missions.  This  is  done 
to  arouse  interest  in  and  to  explain 
programs  being  planned.  The  briefing 


gives  the  goals  of  the  program  and  in¬ 
forms  potential  proposers  for  the 
mission  what  the  program  constraints  are 
likely  to  be.  The  NASA  5  year  plan  is 
one  of  the  major  documents  that  advises 
Industry.  On  the  GRO  program,  an 
Industry  briefing  was  arranged  after  the 
selection  of  the  scientific  instruments. 
This  alerted  industry  to  the  needs  of 
the  scientific  instrumenters  and  what 
portion  of  their  work  was  open  to 
industrial  competition.  It  gave  industry 
an  idea  of  the  physical  requirements 
needed  by  a  spacecraft  to  support  this 
complement  of  instruments. 

3.  Study  Phase  RFP 

a.  General 

The  basic  idea  of  the  study  phase  is  to 
limit  the  government  input  to  the  manda¬ 
tory  constraints  so  that  the  proposers 
will  have  a  maximum  flexibility  in  their 
proposals.  In  most  cases  within  NASA, 
the  mandatory  constraints  essentially 
limit  the  response  of  a  proposer  to  vari¬ 
ations  on  a  known  solution  or  a  fixed 
concept.  The  proposer  does  have  the 
opportunity  to  be  innovative  in  his  ap¬ 
proach,  his  implementation  scheme,  his 
operational  plans,  and  his  cost  control 
methods.  The  Mission  Need  Statement  that 
is  signed  by  the  NASA  Administrator  out¬ 
lines  the  major  constraints  for  a  pro¬ 
curement.  The  normal  constraints  given 
by  the  Mission  Need  Statement  are:  (1) 
a  spacecraft  is  required  to  do  the  mis¬ 
sion,  (2)  the  Shuttle  must  be  used  as  the 
launch  vehicle,  (3)  data  collection,  re¬ 
duction  and  analysis  should  be  done  using 
existing  facilities  to  the  extent  possi¬ 
ble,  and  (4)  the  payload  (scientific  or 
application  instruments)  will  be  obtained 
using  an  AO  and  be  Government  Furnished 
Equipment  (GFE)  to  the  proposer  or  the 
instruments  are  to  be  furnished  by  the 
prime  or  mission  contractor.  These  con¬ 
straints  determine  the  parameters  for  the 
spacecraft,  the  characteristics  of  the 
communication  system  and  the  form  of  the 
data  reduction  and  analysis.  The  pro¬ 
posers  are  given  tliis  conceptual  skelton 
on  which  they  must  build  their  responses 
to  an  RFP.  In  many  NASA  procurements  the 
basic  concept  is  set  by  the  constraints. 
What  is  sought  using  the  NMI  1700. 14A  is 
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a  better  idea  for  the  overall  approach 
and  the  most  economical  use  of  the  re¬ 
sources  available  to  do  the  mission. 


The  study  RKP  is  designed  to  contain  in¬ 
formation  on  the  constraints,  a  simple 
Statement  of  Work,  a  general  set  of  eval¬ 
uation  criteria  and  a  small  list  of 
deliverables.  Evaluation  criteria  will 
not  be  discussed. 


(1)  Constraints 

At  the  present  time  proposers  must  spend 
a  good  deal  of  time  determining  the 
Shuttle,  the  Tracking  and  Data  and  Relay 
Satellite  System  (TDRSS)  and  the  ground 
communications  requirements  since  some 
of  these  systems  are  presently  in  the 
development  phase.  In  a  few  years  the 
handling  of  reqirements  using  those  sys¬ 
tems  will  become  routine.  The  present 
day  RFP  will  reference  the  users  manual 
for  these  systems.  The  proposers  will 
be  expected  to  know  Che  content  of 
these  manuals  and  any  peculiarity  of 
these  systems  which  might  affect  the 
Implementation  of  his  ideas. 


The  instrument  payload  documentation  or 
a  straw  Instrument  payload  description 
is  normally  furnished  as  part  of  the 
data  package  of  an  RFP  since  this  info- 
matlon  is  specific  to  the  RFP  and  not 
general  public  knowledge.  General  docu¬ 
ments  such  as  various  user  guides  or 
parts  lists  are  referenced  since  they 
are  public  documents.  The  payload  in¬ 
strumentation  requirements  and  the  TDRSS 
and  STS  User  Handbook  establish  the 
general  characteristics  that  a  spacecraft 
and  ground  system  must  possess.  Tliere 
will  be  some  trade-offs  between  the 
spacecraft  and  the  ground  svstem  left  to 
the  discretion  of  a  proposer,  however, 
bounds  on  such  items  as  spacecraft  power, 
thermal  design,  pointing  accuracy,  bit 
rales,  operations  controls,  magentic 
restraint  and  ground  systems  response 
time,  and  user  accommodations  will  be 
set . 


(2)  Statement  of  Work 

The  Statement  of  Work  (Attachment  1)  is 
written  to  give  maximum  flexibility  to 


the  proposer.  It  provides  information 
not  found  in  the  constraints  (e.g.,  space¬ 
craft  lile  and  classification  (NMI  801(J 
dated  9/26/79  "Classification  of  NASA  STS 
Payloads")!,  gives  a  general  statement  of 
what  the  mission  is  and  states  what  the 
output  of  the  study  is  to  be  (e.;’,., 
lunctional  specifications  for  the  syst<.n,s 
and  subsystems)  . 

(3)  Deliverable.s 

The  list  of  deliverables  is  held  to  the 
minimum  so  that  maximum  amount  of  re¬ 
sources  can  be  spent  on  the  study  itself. 

A  mid-term  report  and  final  study  report 
are  usually  required.  Study  contracts 
are  normally  fixed  price;  therefore,  no 
financial  reporting  is  required. 

c .  Product  Assurance 

Two  items  that  NASA  will  pay  more  and 
more  attention  to  are  the  value  returned 
for  the  value  invested  and  customer  (sci¬ 
entific  community,  NASA,  etc.)  satisfac¬ 
tion.  An  overall  approach  is  desired — 
one  that  satisfies  the  real  purpose  of 
the  mission,  i.e.,  delivers  the  proper 
data  to  the  appropriate  user  in  a  timely 
manner.  This  focus  on  the  end  product 
has  resulted  in  a  very  high  level  of 
confidence  being  required  of  the  ground 
system . 

Flight  systems  are  expected  to  work. 

After  20  years  of  experience,  first 
flight  units  are  now  being  considered 
part  of  an  operational  system  rather  than 
test  vehicles.  Large  data  handling  and 
communication  .systems  have  reached  matu¬ 
rity.  The  main  difficulty  in  the  past 
has  been  the  fragmented  approach  where 
emphasis  shifts  from  the  flight  hardware 
and  software  to  the  data  operational 
hardware  and  .software  almost  as  a  step 
function.  Now  one  wants  ,i  thorougli  de¬ 
tailed  plan  for  achieving  the  final 
objective,  i.e.,  j’.etting  the  dat.a  to  tiie 
user.  This  plan  should  be  started  in 
the  study  phase  and  permeate  tlte  pl.inning 
of  the  implementation  phase.  Tlie  stating, 
of  the  classification  of  the  paylo.id  in 
the  RFP  informs  tlie  proposiTs  of  approxi¬ 
mately  what  level  of  quality  .issurinre 
will  be  required,  wh.it  reviews  .ire  to  he 
held  .and  wh.at  doi  umeiit at  ion  wi  1  be  n  - 
quired.  This  pavload  e  lassi  1  it  at  ion 


(2)  Statement  of  Work 


system  is  new  and  most  of  the  implement¬ 
ing  documentation  is  in  the  formative 
state.  The  documentation  should  be 
available  by  the  latter  part  of  1980. 

4 .  Implementation  Phase  RFP 

a .  General 

The  GSFC  approach  to  th^  Implementation 
phase  RFP  is  to  use  general  functional 
specifications  for  the  mission  and  let 
the  proposer  respond  with  specific 
functional  specifications  for  the  sys¬ 
tems  and  subsystems.  The  goal  is  to 
give  the  proposer  the  greatest  degree 
of  freedom  possible  in  his  proposal. 

The  RFP  does  get  specific  in  the  quality 
assurance  requirements,  external  inter¬ 
face  requirements,  and  financial  and 
progress  reporting  requirements. 

b.  Content  of  the  Implementation 

Phase  RFP 

The  implementation  phase  RFP  contains  a 
detailed  Statement  of  Work,  a  detailed 
list  of  mandatory  Interfaces,  a  list  of 
quality  assurance  documents  a  specific 
set  of  evaluation  criteria  and  a  de¬ 
tailed  list  of  deliverables.  Evaluation 
criteria  will  not  be  discussed. 

(1)  Constraints 

The  Interface  and  quality  assurance  docu¬ 
ments  are  the  real  specifications  for  the 
work  to  bo  done  and  are  the  constraints 
for  this  RFP.  These  documents  establish 
the  major  constraints  in  detail  and  limit 
a  proposer  to  a  definite  set  of  rules 
governing  his  overall  plans. 

The  payload  interfaces  set  the  basic  re¬ 
quirements  for  the  spacecraft  and  ground 
system  design.  The  launch  vehicle  re¬ 
quirements,  incl'.dlng  the  methods  used 
to  determine  tariff,  will  drive  the 
mechanical  design  and  operational  plans. 
The  communication  link  Interfaces  will 
determine  the  major  specification  for 
data  handling  systems  for  both  the  space¬ 
craft  and  the  ground  systems.  The 
quality  assurance  documents  will  state 
how  the  systems  arc  to  be  built,  tested 
and  operated  to  assure  success. 


The  Statement  of  Work  for  the  implementa¬ 
tion  phase  RFP  is  essentially  a  list  of 
all  the  tasks  to  be  performed  and  the 
work  to  be  accomplished  under  each  task 
(Attachment  2.  a  page  excerpt  from  the 
Earth  Radiation  Budget  Satellite  RFP). 
Each  task  is  defined  by  a  general  state- 
emnt  and  a  statement  of  responsibilities. 
These  inform  the  propc:.er  that  he  is 
responsible  for  all  work  to  be  done  to 
accomplish  the  task  even  if  such  work  is 
not  specifically  stated.  One  of  the 
primary  purposes  of  the  Statement  of  Work 
is  to  define  in  concise  and  clear  terms 
the  responsibilities  of  all  parties. 

This  is  normally  done  by  using  rules  of 
exception,  i.e.,  the  proposer  is  respon¬ 
sible  unless  specifically  told  that  he 
is  not  responsible.  The  Statement  of 
Work  takes  about  20  to  30  pages.  It 
consists  mainly  of  very  short,  one  line 
descriptive  sentences. 

(3)  Deliverables 

This  will  be  a  detailed  list  of  all  items 
needed.  An  attempt  will  be  made  to  limit 
the  type  and  amount  of  reporting,  tech¬ 
nical  documentation  and  other  items  to 
conserve  resources. 

c.  Product  Assurance 

The  role  of  the  Source  Evaluation  Board 
(SEB),  the  Technical  Advisory  Committee 
(TAG),  and  the  Business  Management  Com¬ 
mittee  (BMC)  in  a  two  phase  procurement 
under  NMI  1700. 14A  is  not  only  to  prepare 
the  RFP  and  evaluate  the  proposal  but  to 
monitor  the  work  being  done  under  the 
study  phase. 

One  goal  of  NASA  is  to  maximize  the 
utility  of  its  resources.  Each  new  major 
space  mission  requires  significant  re¬ 
sources  and  a  failure  of  a  mission  would 
probably  result  in  putting  the  mission 
back  to  the  starting  point  of  justifying 
its  existence.  It  is  very  important  that 
every  effort  be  made  to  have  a  successful 
mission  consistent  with  the  guidelines 
Issued  in  the  classification  of  NASA 
Shuttle  payloads  (Attachment  3,  a  very 
preliminary  sample  of  STS  Minimum  Assur¬ 
ance  Requirements) . 
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i -  Scope 

This  Statement  of  Work  covers  the  procurement  of  the  study  phase  (Phase  I) 
of  a  two-phase  procurement  of  the  Gamma  Ray  Observatory  (GRO)  in  accordance 
with  the  Office  of  Management  and  Budget  Circular  No.  A- 109  and  implemented 
by  NASA  Management  Instruction  (NMI}  7100. 14A  dated  April  19,  1978.  In  the 
study  phase,  the  contractor  is  to  present  his  concepts  for  the  GRO  mission. 

1 1 .  Gamma  Ray  Observatory  System  Baseline  Requirements 

The  GRO  system  consists  of  two  segments:  the  free  flyer  segment  and  the 
ground  segment.  The  contractor  is  responsible  for  all  the  free  flyer  hard¬ 
ware  and  software  except  the  scientific  instruments  hardware  and  software 
which  will  be  Government  Furnished  Equipment  (GFE).  The  contractor  is 
responsible  for  all  the  mission  unique  ground  segment  hardware  and  software 
required  to  support  the  free  flyer  segment  from  inception  to  the  end  of 
mission  life  except  the  scientific  instruments  ground  support  systems  which 
will  be  GFE. 

The  free  flyer  segment  is  to  be  launched  by  the  Shuttle  into  a  near  circular 
orbit.  The  free  flyer  will  have  a  2-year  mission  life.  The  free  flyer  will 
be  capable  of  being  retrievable.  The  free  flyer  segment  must  be  compatible 
with  the  TDRSS,  the  Shuttle,  the  NASA  ground  systems,  and  the  five  scientific 
instruments  listed  below: 

1.  0.1-10  MeV  Gamma  Ray  Instrument — Naval  Research  Lab,  Dr.  Kurfess 

2.  Compton  Telescoper  Instrument — Max-Planck-Insti tute.  Dr.  Schonfelder 

3.  Gamtia-Ray  Spectroscopy  Instrument — Univ  of  Calif,  San  Diego, 

Dr.  Peterson 

4.  High  Energy  Gamna-Ray  Telescope  Instrument — Goddard  Space  Flight 
Center,  Dr.  Fichtel 

5.  Gamma-Ray  Burst  Instrument — Marshall  Space  Flight  Center,  Dr.  Fishman 

The  ground  segment  consists  of  all  mechanical,  electrical,  and  software 
systems  required  to  support  the  free  flyer  segment  during  construction, 
integration,  test,  and  pre  and  post-launch  operations.  The -contractor  will 
operate  the  spacecraft  in  orbit. 


Attachment  1 
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III.  Phase  I  Definition 
Statement  of  Work 


In  the  system  design  concept  study,  the  contractor  is  to  develop  his 
proposed  mission  concept  and  explain  how  it  satisfies  the  objectives  of 
the  GRO  mission.  He  is  to  illustrate  how  each  individual  segment  of  his 
concept  fits  into  the  overall  plan  to  satisfy  the  mission's  needs  and  how 
he  would  economically  apply  the  resources  to  minimize  duplication  of  effort 
and  reduce  costs  in  the  hardware,  software,  and  operational  phases  of  over¬ 
all  concept.  He  will  perform  the  necessary  analyses  and  trade-off  studies 
needed  to  demonstrate  the  validity  of  his  concept. 

The  contractor  is  to  refine  his  concept  of  the  flight  segment.  He  is  to 
illustrate  his  approaches  to  handling  all  external  interfaces,  such  as  the 
scienti/ic  interfaces,  the  STS,  the  TDRSS,  and  the  POCCNET.  He  is  to 
define  his  general  test  plan,  safety  plan,  and  quality  assurance  plan  out¬ 
lining  the  detailed  content  he  intends  to  develop  for  the  actual  implemen¬ 
tation.  He  is  to  discuss  the  studies  he  has  done  to  validate  his  concept 
and  outline  additional  studies  and/or  trade-offs  he  plans  for  the  execution 
phase  and  how  he  plans  to  make  the  free  flyer  retrievable. 

The  contractor  is  to  explain  how  he  intends  to  implement  the  ground  segment 
of  the  GRO  mission.  His  general  plan  to  use  the  ground  segment  hardware, 
software,  and  procedures  during  integration,  test,  prelaunch  operations, 
launch,  and  post-launch  operations  is  to  be, explained.  He  is  to  explain  bcth 
the  capabilities  and  the  limitations  of  his  proposed  ground  segment.  He  is 
to  show  how  this  segment  meets  the  overall  mission  needs. 

Alternates 


The  contractor  is  to  make  recommendations  regarding  the  desirability  or  lax  ' 
thereof  of  for  (1)  servicing  the  Observatory  in  oroit,  (2)  refurbishing 
Observatory  ip  orbit,  and  (3)  providing  controlled  re-entry  of  the  observa¬ 
tory  without  use  of  the  shuttle. 


He  is  free  to  propose  other  alternates  which  he  feels  will  enhance  the 
mission,  either  through  greater  scientific  return  or  significant  cost  benefit 
to  the  government. 
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3.3  SPACE  SEGMENT  REQUIREMENTS 
3.3.1  System  Engitieerinc 

The  contractor  shall  perform  all  systems  engineering  tasks 
required  for  the  successful  performance  of  the  mission.  Including, 
but  not  limited  to,  the  following: 

a.  Analysis  and  formulation  of  the  overall  system  performance 
compatibility,  interactions,  and  margins  requir^riients . 

b.  Develop  design  requirement  specifications  for  system, 
subsystems,  and  components. 

c.  Analyses  to  verify  subsystem  performance,  such  as  thermal 
analysis,  structural  analysis,  control  system  computer  simulations, 
energy  balance  capability;  also  parametric  studies  directed  toward 
minimization  of  instrument  loads  and  stresses. 

d.  System  design  verifications,  including  STS  Safety  require¬ 
ments  verification. 

e.  Evaluation  of  technical  adequacy  of  systems,  subsystems, 
and  equipment  interfaces,  including  evaluation  of  failure  modes 

f.  System  engineering  function/overview  throughout  the  program. 

g.  Definition  and  control  of  system,  subsystem,  and  component 
interfaces,  including  those  with  the  Shuttle,  TDRSS,  GSTDN,  POCC , 
and  instruments. 

h.  Review  all  activities  to  assure  contamination  and  environ¬ 
mental  controls  are  implemented  throughout  the  contract. 

i.  Perform  analyses  to  ensure  that  the  observatory  design 

accommodates  all  reouired -interfaces  and  design  characteristics, 
including  dynamic  inputs  from  tnc  instrument  tomplcment. 

j.  Reporting  and  control  of  system  parameters,  such  as  power 
budget,  mass  properties,  telemetry  and  command  handbook,  mission 
timelines,  etc. 

0.3. 1.1  Directed  Systems  Studies 

The  contractor,  in  addition  to  performing  those  studies  required 
to  accomplish  the  systems  engineering  tasks  specified  above, 
shall  perform  the  following  studies; 
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ST.S^Prel1m1n»r.y  JUnimum  Assurance  Requirements  REVISED 

Dtscviminatort  Between  Payload  Classes 


r  — 

>•1  lap. 

DISCIPLINE 

CLASS  A 

CLASS  B 

CLASS  C 

CLASS  D 

• 

■X 

Flight  Assurance  Reviews 

Number  of  reviews 

Full  cycle  of  six 
reviews 

Full  cycle  of  six 
reviews 

Five  reviews  (System 
concept  rev.  optional) 

Preliminary  Design  Rev. 
Pre-Integration  Review 
primarily  safety  orient 

!-■  -3 

Performance  Verification 

L-  . 

i 

a.  Payload 

1.  Functional  tests 

All  mission  modes 

All  mission  nwdes 

Primary  and  safety 
related  mission  modes 

Safety  related  mission  h 

• 

-■ 

2.  Ln<J-Lo-cri(J  tests 

3.  Structural  Loads 

Al 1  mission  modes 

Analysis 

All  mission  modes 

Analysis 

Primary  and  safety 
related  mission  m^es 
Analysis 

Safety  related  mission  t 

Analysis  for  safety  t 

STS  restraints 

f 

A.  Vibroacoustic 

Acoustic  test 

No  requirement 

No  requlremert 

No  requirement 

- 

1 

5.  EMC 

All  tests 

All  tests 

Less  critical  tests 
dropped 

Safety  related  tests 

6.  Vacuum  Thermal, 
i  Humidity 

a.  Thermal -Vacuum 

Test  at  highest 
practicable  level  of 
assembly 

Test  at  highest 
practicable  level  of 
assembTv 

Thermal  cycling  may 
replace  thermal -vacuum 

If  no  vacuum 
sensitivity  exists 

No  requirement 

1 

b.  Thermal  Balahce 

Test  at  highest 
practicable  level  of 
assembly 

Test  at  highest 
practicable  level  of 
assembly 

Test  where  significant 
performance  uncertainties 
exist 

No  requlranent 

■■ 

c.  Temperature, 
Humidity  (Ocorbit 
descent  B 
landing) 

Analysis 

Analysis 

Analysis  where 
significant  performance 
uncertainties  exist 

No  requirement 

1 

d.  Contamination 

Analysis 

Analysis 

Analysis  for  constraints 

Analysis  for  STS  constra 

e.  Leakage 

Test 

Test 

Consider  test 

No  requirement 

MISSION  ASSURANCE  AND  PROPOSAL 

PREPARATION 

L.  Gomberg 

Manager,  Military  Satellite  Programs 
J.R.  Staniszewski 
Program  Manager,  Navy  Navigation 
Satellite  Program 

Virtually  all  system  level  RFP's  contain 
a  set  of  Mission  Assurance  requirements  - 
some  of  them  quite  explicit,  the  rest 
implicit.  The  explicit  requirements  are 
embodied  in  reference  documents  such  as 
MIL-Q-9858A,  MIL-HDBK  217C,  MIL-STD-1543, 
MIL-STD-721,  MIL-STD-965,  SAMSO  Exhibit 
73-2C,  MIL-S-52779,  MIL-STD-483,  and  MIL- 
STD-1540A.  Most  of  you  are  familiar  with 
these  documents  and  know  how  to  write  a 
proposal  responsive  to  these  requirements 
and  do  not  require  guidance  in  these 
areas.  The  implicit  requirements  are 
much  more  difficult  to  recognize,  and 
consequently  are  more  difficult  to  ad¬ 
dress.  The  difficulty  arises  because  a 
good  system  response  requires  the  inter¬ 
action  between  several  disparate  groups 
and  the  Mission  Assurance  activity  (i.e.. 
Systems  Engineering,  Program  Management, 
and  Cost  and  Pricing).  This  is  further 
confounded  by  organizational  structure 
and  personalities. 

In  preparing  the  proposal  response,  it  is 
important  that  the  issue  of  "tailoring" 
be  addressed.  Tailoring  can  result  in  a 
cost-effective  proposal,  but  extreme  care 
should  be  taken  to  avoid  the  risk  of  be¬ 
ing  considered  non-responsive.  Therefore, 
the  proposer  should  suggest  that  certain 
specification  requirements  be  "tailored" 
to  provide  the  equivalent  mission  assur¬ 
ance  at  lower  cost  or  to  provide  increased 
mission  assurance  at  equivalent  cost. 

If  for  reasons  of  a  win  strategy,  even 
such  tailoring  is  considered  to  be  too 
risky,  such  suggestions  can  be  offered  in 
the  form  of  alternate  lower  cost  but 
mission-effective  proposals.  Such  alter¬ 
nate  proposals  are  usually  allowed  and 
encouraged  by  the  RFP. 

Mission  Assurance  is  involved  in  all 
program  phases,  particularly  in  that  of 
proposal  preparation.  This  involvement 
and  its  shift  in  emphasis  and  skill 
requirements  as  in  the  program  evolves  are 
illustrated  in  Figure  1. 


The  degree  of  participation  and  the  Mission 
Assurance  organizational  relationship 
must  be  clearly  delineated  by  the  con¬ 
tractor  to  provide  the  customer  with  the 
confidence  that  the  proposed  system  will 
be  effective  and,  if  implemented,  have  a 
high  probability  of  success  (See  Figure  2). 

It  is  imperative  that  Mission  Assurance 
become  involved  early  in  the  proposal 
effort  because  the  decisions  affecting 
basic  system  architecture  are  made  in  the 
early  stages  of  the  effort.  Once  the 
architecture  is  established  and  detailed 
configurations  and  requirements  are  appor¬ 
tioned,  cost  estimating  can  begin.  In 
this  phase.  Mission  Assurance  provides 
reliability  models,  mean-mission-duration 
calculations,  life  and  parts  degradation 
information,  failure  modes  and  effects 
analysis  (FMEA),  single  point  failure 
identification,  and  a  preliminary  hazards 
analysis.  Systems  Engineering  uses  all  of 
these  factors,  among  others,  to  develop 
and  define  the  system.  The  Mission  Assur¬ 
ance  contributions  are  illustrated  by 
Figure  3,  Figure  4  (reliability  model). 
Table  1  (typical  failure  rates),  and 
Table  2  (P(s)  calculations).  The  develop¬ 
ment  of  this  information  is  an  iterative 
process,  requiring  continual  interface 
between  Systems  Engineering,  Mission 
Assurance,  and  Management.  It  is  bene¬ 
ficial  during  the  System  Definition  Phase 
to  co-locate  these  activities  to  encourage 
good  communications. 

Mission  Assurance  has  developed  several 
mathematical  techniques  to  make  system 
trades.  These  include  Markov  analysis 
(see  Figure  5),  optimum  analysis  using 
dynamic  programming  (see  Figure  6),  and 
Monte  Carlo  simulations  (see  Figure  7). 

All  play  a  role  in  the  Life  Cycle  Cost 
(LCC)  process. 

Mission  Assurance  is  not,  as  generally 
supposed,  divorced  from  the  cost  esti¬ 
mating  process.  In  fact.  Mission  Assur¬ 
ance  has  a  significant  role  to  play  in 
this  area. 

The  trend  in  recent  RFP's  for  operational 
systems  is  for  the  customer  to  specify  an 
availability  and  the  total  system  life, 
and  the  contractor  to  propose  a  total 
system,  including  the  logistics,  opera¬ 
tions,  number  of  satellites,  spares,  and 
design  life.  Estimates  of  mean-mission 
duration  (MMD),  probability  of  survival 


(Pj),  design  life,  and  mean  time  to 
repair  are  made  and  used  to  develop  total 
cost  of  ownership  (TCO)  using  life-cycle 
cost  models  (LCCM).  An  imaginative 
approach  in  this  area  can  have  a  dramatic 
impact  on  cost  and  in  the  probability  of 
a  win.  Figures  8,  9,  and  10,  illustrate 
a  typical  satellite  design  as  a  function 
of  parts,  level,  and  design  life, 
respectively. 

An  important  parameter  appearing  more  and 
more  in  current  RFP's  is  system  effective¬ 
ness,  which  relates  reliability,  avail¬ 
ability,  and  performance.  It  can  be 
expressed  as  E  =  P*R*A.  The  performance 
parameter  is  somewhat  subjective;  however, 
it  may  not  necessarily  be  unity  as  some 
people  may  argue.  For  example,  if  a 
system  may  require  several  different  sen¬ 
sors  for  different  customers,  it  may  not 
be  possible  to  serve  all  of  the  users  with 
a  single  platform.  Comprises  are  required 
which  result  in  a  performance  index  which 
is  less  than  unity  (P  1).  E  can  be 
calculated  quickly  with  the  nomograph 
shown  in  Figure  11. 

Another  important  application  of  the 
Mission  Assurance  discipline  is  in  defin¬ 
ing  and  negotiating  contract  profit  incen¬ 
tives.  There  is  a  trend  requiring  the 
contractor  to  accept  negative  incentives  - 
meaning  that  the  penalty  for  early  mission 
failure  is  to  dilute  profit  and  to  go 
into  the  basic  cost.  With  FFP  contract, 
this  can  be  very  painful. 

Mission  Assurance  must  be  involved  with 
the  definition  of  failure,  mission  suc¬ 
cess,  and  partial  success  modes.  Addi¬ 
tionally,  there  is  a  desire  on  the  part 
of  mangement  to  reduce  risk  by  insuring 
the  incentive.  The  insurance  companies 
require  that  the  P(s)  and  MMD  be  calcu¬ 
lated  for  all  mission  modes,  and  that  the 
derivation  for  these  failure  rates  be 
provided.  The  insurance  companies  under¬ 
stand  and  appreciate  these  estimates 
because  of  their  actuarial  expertise. 

They  also  want  to  see,  and  it  is  desirable 
to  show,  the  reliability  growth  curves 
on  proposed  systems  which  are  derivatives 
of  existing  systems.  Good  presentations 
in  this  area  have  big  payoffs  in  both 
the  contract  incenti vizing  and  on  the 
insurance  premium. 

With  the  advent  of  the  space  shuttle  (STS) 
era,  a  new  dimension  has  been  added  to 


satellite  system  configuration  develop¬ 
ment.  For  the  first  time  it  will  be  pos¬ 
sible  to  retrieve  failed  satellites,  or 
even  repair  them  in  orbit.  Such  goals 
are  not  as  easily  achieved  as  the  media 
believes:  nonetheless,  retrievabi 1 ity  is 
feasible.  This  results  in  Availability 
assessments  for  STS  era  satellites  in 
which  Mean  Time  to  Replacement  (MTTR)  can 
truly  include  repair  time  rather  than 
being  limited  to  total  replacement  strat¬ 
egies  of  the  Expendable  Launch  Vehicle 
era.  Elaborate  models  have  been  con¬ 
structed  using  mission  assurance 
techniques  (see  Figures  12,  13,  and  14). 
These  show  that  implementing  a  retrieva- 
bility  capability  pays  only  if  the  payload 
has  a  relatively  low  reliability  and  is 
very  expensive.  However,  one  cannot 
prejudge  the  outcome,  and  detailed  models 
must  be  constructed  and  various  cases 
run  for  each  proposed  system. 

Another  impact  of  the  STS  on  systems 
design  is  the  issue  of  replacement 
strategy.  It  now  appears  that  scheduling 
for  space  on  an  early  STS  will  be  very 
difficult  and  costly.  As  such,  the 
question  of  "launch  on  deman"  versus 
storage  "on  orbit"  must  be  addressed. 

The  expendable  launch  vehicle  (ELV) 
provided  the  user  with  a  great  deal  of 
flexibility.  To  maintain  the  satellite 
constellation  in  the  STS  era,  one  can 
schedule  space  on  the  STS  on  a  regular 
basis  and  cancel  if  it  is  not  required; 
however,  this  approach  will  be  very 
costly.  Storing  in  orbit  will  support 
the  system  availability;  however,  it 
subjects  the  satellite  to  degradation. 
Mission  Assurance  techniques  must  be 
utilized  to  provide  estimates  of  MMD, 
life,  and  degradational  factors  for  this 
on  orbit  storage  option. 

In  the  STS  era,  system  safety  is  a  major 
issue  —  it  is,  in  fact,  the  paramount 
consideration.  Figure  15  shows  a  matrix 
of  safety  requirements.  The  proposal 
effort  requires  that  a  top-level  hazards 
analysis  be  performed,  as  illustrated  in 
Table  3. 

The  Mission  Assurance  activity  is  also 
deeply  involved  in  the  parts  procurement 
program.  Parts  requirements  are  signifi¬ 
cant  cost  drivers,  and  must  be  viewed  in 
the  context  of  total  system  cost  as 
illustrated  in  Figure  8,  9,  10,  and  13. 


A  screening  matrix  such  as  the  one  shown 
in  Figure  16  helps  to  avoid  ambiguity, 
and  is  also  useful  for  estimating  costs. 

Basically  the  message  to  space  system 
contractors  is  simple:  in  addition  to 


explicit  requirement  delineated  in  the 
RFP,  there  are  the  many  implicit  require 
ments  which  must  be  considered  during 
evolution  of  the  system  baseline,  and 
are  an  essential  factor  in  a  successful 
RFP  response. 


TABLE  1. 


PART  FAILURE  RATES  FOR  SPACE  VEHICLE 
SYSTEM  PRELIMINARY  RELIABILITY  ESTIMATE 


X  (xio-9)* 

Failures/Hour 


ITEM 

1 

Integrated 

Bipolar  Digital  (TTL4DTL) 

Circuits 

1 

1-20  gates 

21-50  gates 

51-100  gates 

Beam  Lead,  ECL,  Bipolar 
i  MOS  Linear,  Other  MOS 

1-20  gates 

21-50  gates 

51-100  gates 

Linear  32  transistors 

Linear  33-100  transistors 

Transistors 

Si  NPN 

Si  PNP 

FET 

Unijunct ion 

Diodes 

Si  General-Purpose 

Zener  &  Avalanche 

Varactor,  Step 

Capacitors 

Ceramic  CKR 

Tantalum  Solid  CSR 

Tantalum  Non-Solid  CLR 
Paper/Plastic  CHR  etc. 

Glass  CYR 

Variable  Piston  PC 

Filter  (EMI  Feedthrough) 

Mica  CMR 

Resistors 

Composition  RCR 

Film  RNR 

Film  RLR 

Wlrewound  Power  RMR 

Wlrewound  Chassis  Mount  RER 
Variable  Wlrewound  RTR 

Variable  Non-Wlrewound  RJR 
Thermistor 

Inductive 

Power  Transformer/Colls 

RF  Transformer/Coils 

Connectors  Circular,  Rack  &  Panel,  PC 

Relays  Latching 

1.4 

19.3 

10.0 

20.0 


30.9/15.3 

33/17.4 


*  These  failure  rates  were  used  for  calculation  of  the  Space 
Vehicle  System  Preliminary  Reliability  Estimate,  and  are  in 
accordance  with  MIL-HDBK-217B. 


TABLE  2.  PRELIMINART  SYSTEM  RELIABILITY  ESTIMATE 


EQUIPMENT 

PROBABILITY  OF  SURVIVAL  (Pj) 
FOR  1  YEAR  ON-ORBIT 

STRUCTURE  &  THERMAL  CONTROL 

0.9989 

ATTITUDE  CONTROL 

0.984S 

electrical  power  &  DISTRIBUTION 

0.9R7S 

COMMAND  GROUP 

0.9840 

COMMUNICATIONS  GROUP 

0.9993 

DATA  PROCESSING  GROUP 

0.9750 

TELEMETRY 

0.9845 

SPACE  VEHICLE  SYSTEM 

PRELIMINARY  RELIABILITY 

ESTIMATE 

0.9166 

TABLE  3.  TYPICAL  PRELIMINARY  HAZARD  LIST^^^  FOR  SPACECRAFT/STS  INTERFACE 


Subsystem/ 

Function 

Hazard  Group 
(JSC  13830) 

Hazard  Event/Description 

Applicable  Safety 
Requirement 
(NHB  1700.7) 

Electrical 

Corrosion 

Battery  (Nickel-Hydrogen)  leakage. 

Action  of  electrolyte  if  leak  goes 
undetected  for  some  time. 

209-1,  208-4 

Electrical 

Shock 

High  voltage,  associated  with  exoosed 
connections  or  brought  to  exoosed 
surface  as  result  of  a  short. 

213,  206 

Explosion 

Battery,  develons  excessive  internal 
oressure  as  a  result  of  internal  short 
or  excessive  temoerature. 

Note:  The  battery  normal  ooerating 
oressure  is  in  the  800  psi  range,  with 
case  designed  to  a  3000  osi  level. 
Develooment  of  excessive  overoressure 
would  only  result  from  a  series  of 
catastroohic  events  oossibly  including 
case  damage  that  would  facilitate 
ruoture  at  below  design  levels.  Hazard 
reduction  can  be  readily  accomolished. 

213,  209-1 

Fire 

Battery  release  of  hydrogen  due  to 
overcharge,  electrical  short,  or  damage 
to  cell  case  or  seal. 

209-1,  209-3, 

213 

Environmental 

Control 

Temperature 

Extremes 

Heater  malfunction  that  develoos  hot 
snot  temoeratures  sufficient  to  itjnitc 
combustibles,  or  to  develop  system 
overoressures  roieasinq  combustible 
liauids  or  qasses. 

213,  209-4 

Materials 

Contamination 

Release  of  outgassing  products  or 
oarticiilate  matter  during  launcti  or 
landing . 

209-4,  209-1 

Corrosion 

Release  of  corrosive  materials  as  a 
result  of  battery  or  oropulsion  (fuel! 
leaks  (see  aoplicable  subsystem  Hazard 

List)  or  release  of  corrosive  outgassing 
or  secondary  material  of  corrosive 
nature  as  a  result  of  temoerature/time 
excursions . 

209-1,  209-4 

Fire 

Flammable  material  in  contact  with  high 
temoerature  ooint  source  or  in  contact 
with  fuel  spillage. 

209-1,  209-3 

content  of  this  table  is  compatible  with  JSC  Form  542A  (Feb.  78)  which  is  the 
documentation  format  required  by  JSC  13830. 


TABLE  3.  TYPICAL  PRELIMINARY  HAZARD  LIST  FOR  SPACECRAFT/STS  INTERFACE  (Continued) 


Subsystem/ 

Function 

Hazard  Group 
(JSC  13830) 

Hazard  Event/Description 

Applicable  Safety 
Requirement 
(NHB  1700.7) 

Mechanical 

Collision 

Failure  of  orimary  soacecraft  structure 
durinq  launch  or  landing  loads. 

208-1,  208-2, 

208-3,  213 

Failure  of  cradle  to  maintain  structural 
integrity  durinq  launch,  landing,  or 
deployment . 

Inadvertent  extension  of  solar  arrays, 
antennas  or  booms  prior  to  orbiter 
clearance  minimums. 

Pressure 

Systems 

Corrosion 

Pressurized  lines  and  fittings  develop 
small  leaks  releasing  hydrazine. 

208- 4,  208-5, 

209- 1 

Battery  leaks  develoo  as  result  of  over¬ 
pressure  or  seal  malfunction  (see 

Electrical  Subsystem) . 

Stress  corrosion  of  metallic  fittings, 
nuts,  etc. 

Explosion 

Rupture  of  propellant  lines,  and  tan)cs 
during  launch,  deployment,  or  emergency 
abort . 

208- 4,  208-5, 

209- 1,  209-3, 

202-2b 

See  Battery  generated  exnlosion  ootential 
under  'Electrical  Subsystem  Hazard  List.' 

Propulsion 

Collision 

Inadvertent  ignition  of  prooulsion 
subsystem. 

209-1 

Explosion 

Rupture  of  prooellant  lines  and  tan)cs 
durinq  abort,  launch,  and  deployment 
(see  'Pressure'  subsystem  Hazard  List. 

202-2b 

Fire 

Leakage  of  hydrazine  from  lines  and 
fittings  or  release  of  battery  generated 
hydrogen  (see  'oressure'  and  'electrical' 
subsystems) . 

209-1,  209-3, 

213 

Pyrotechnics 

Collision 

Inadvertent  firing  of  an  initiator  causes 
prooulsion  ignition,  deployment  of 
extendable  device,  or  release  of  soace* 
craft  from  cradle. 

210-211 

212-2 

Explosion 

Same  as  above  except  that  result  is  not 
translated  into  spacecraft  motion. 

Radiation 

Radiation 

RF  radiation  escapes  into  orbiter  bay 
during  orelaunch,  launch  and  pre-doors 
ooen  phase  due  to  inadvertent  turn  on 
of  transmitters  or  seal  leakage. 

212-2 

Structures 

Collision 

Failure  of  orimary  structure  (spacecraft 
or  cradle)  causes  imoact  with  the  orbiter 
bay  walls  or  bay  door  as  a  result  of 
severe  deflection.  Failure  could  occur 
during  launch,  abort,  deoloyment,  or 
landing  (standard  or  emergency) . 

208-1,  208-2, 

206-3 
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Figure  1.  Mission  Assurance  Involvement  in  Space  System  Life  Cycle 
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Figure  2.  Mission  Assurance  Activities 
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NOTE:  MEAN  MISSION  DURATION  WAS  CALCULATED  IN  ACCORDANCE  WITH  ' 
MILSTD-15431USAF).  j 


Figure  3.  Mean  Mission  Duration  vs.  Truncation  time 
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Figure  4.  Satellite  Reliability  Model,  On-Orbit  Mission  Phase  (cont) 

580 


581 


^  Figure  16.  Parts  Screening  and  Inspection  Matrix 

I 


584 


CONTRACTOR  PRE-SUBMITTAL 
PROPOSAL  PREPARATION 
AND  EVALUATION 

H.  Tom  Morri s 

CONTRACTS/PROPOSAL  MANAGEMENT 
SPACE  OPERATIONS  &  SATELLITE  SYSTEMS  DIV. 
ROCKWELL  INTERNATIONAL  CORPORATION 


Although  the  ultimate  responsibility  for 
evaluating  proposals  lies  with  the  buyer, 
sellers  must  also  evaluate  their  proposal 
before  submittal  to  not  only  assure 
themselves  that  an  achievable  and  reason¬ 
able  approach  can  be  developed  but  also 
that  the  best  possible  presentation  is 
being  made.  Initially,  an  evaluation  of 
the  RFP  requirements  must  be  made  to 
determine  if  seeking  the  business  is  in 
the  best  interests  of  the  company  and  if 
the  resources  (technical,  financial, 
facilities,  etc.)  are  available  should  a 
contract  be  awarded.  If  the  determin¬ 
ation  is  made  that  the  contract  should 
be  pursued,  a  proposal  must  be  prepared 
keeping  in  mind  the  buyer's  requirements 
and  concerns. 

I .  BID  DECISION 

At  the  outset  of  the  bid  decision 
process  a  seller  must  decide  if  the 
type  of  contract  and  program  risk  are 
consistent.  There  are  times  when  we 
will  decide  not  to  pursue  a  contract 
because  of  an  unw i I i i ngness  to  accept 
the  contract  terms.  Sometimes  this 
conclusion  is  reached  during  the 
proposal  preparation  cycle.  Much  of 
our  business  is  research  and  develop¬ 
ment  work  which  does  not  lend  itself  to 
fixed  price  type  contracting.  There¬ 
fore,  if  we  decide  to  seek  the 
business  we  must  find  ways  of  minimiz¬ 
ing  our  exposure.  We  have  concluded 
that  an  important  factor  in  success¬ 
fully  accomplishing  this  goal  is  to 
have  a  strong  and  effective  Assurance 
Management  organization.  They  are 
chartered  with  finding  ways  of 
increasing  probability  of  mission 
success  at  minimum  costs.  As  indi¬ 
cated  later,  this  organization  is 
an  important  part  of  the  red  team. 


If  they  conclude  that  necessary 
program  deciplines  (e.g.  high 
reliability  parts)  cannot  be  accom¬ 
plished  within  the  allocated 
resources,  a  decision  must  be  made  as 
to  the  available  tradeoffs  and  how 
the  ultimate  approach  selected  will 
meet  the  buyer's  objectives. 

II.  EVALUATION  PROCESS 

Prior  to  final  approval  and  submittal 
of  the  proposal,  we  usually 
have  a  "red  team"  evaluate  the 
proposal.  This  team  consists  of 
individuals  who  become  familiar  with 
the  requirements  specified  in  the 
RFP  and  SOW,  but  are  not  involved  in 
the  actual  proposal  preparation,  and 
are  knowledgeable  of  the  capabilities 
of  the  company.  It  is  their  Job  to 
not  only  assure  that  from  the 
company's  standpoint  the  proposal 
makes  sense,  but  that  the  require¬ 
ment  established  in  the  RFP/SOW  is 
being  fully  met  in  the  most  desirable 
manner.  To  do  this  requires  a 
detailed  review  by  several 
Individuals.  Assignments  are  made 
to  review  technical  approaches  to 
assure  that  they  are  not  only  feas¬ 
ible,  but  provide  for  low  risk 
programs.  When  non-recoverab I e 
satellites  are  involved,  assurance 
that  mission  objectives  will  be  met 
is  of  primary  concern.  It,  there¬ 
fore,  becomes  extremely  important 
to  be  able  to  demonstrate  that  not 
only  the  design  concept  meets  mission 
requirements  but  that  a  management 
system  exists  to  provide  the 
diciplines  needed  to  produce  a 
rel iable  product. 

In  reviewing  the  proposal  presen¬ 
tation  and  evaluating  whether  it  will 
"sell",  the  red  team  looks  for 
several  things  that  will  make  the 
proposal  more  effective. 

These  are; 

1 )  Does  your  story  come  across 
early  and  spark  the  readers 
i nterest? 

2)  Does  the  message  emphasize  the 
buyers  gain  in  what  is  being 
proposed  and  the  advanfaqes  of 
the  proposed  approach? 
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EVALUATION  PROCESS  (continued) 

3)  Is  the  presentation  specific  in 
its  approach? 

4)  Is  the  presentation  written  in 
a  positive  style? 

5)  Is  the  proposal  written  in  a 
manner  that  the  buyer  wi I  I 
understand  all  the  details  of 
the  proposed,  technique  and  the 
terminology  used? 

6)  Is  the  presentation  brief  and 
to  the  point? 

7)  Is  related  experience  clearly 
set  forth? 

To  assist  our  red  teams  a  Proposal 
Checklist  is  utilized  similar  to 
the  one  set  forth  in  Section  III. 

Each  red  team  chairman  will  tailor 
it  to  meet  his  own  needs. 

Normally,  the  red  team  consists  of 
key  members  of  the  technical  and 
administrative  staffs  and  may,  if 
appropriate,  include  major  sub¬ 
contractors.  One  of  the  members 
represents  Assurance  Management  and 
is  responsible  for  evaluating  the 
technical  design  aspects  that  deal 
with  mission  assurance,  such  as 
reliability,  system  safety,  and 
configuration  control  as  well  as 
the  production  aspects  including 
manufacturing  processes,  parts 
control,  and  inspection  procedures. 
Each  member  reviews  his  assigned 
area  and  reports  to  the  team  Chair¬ 
man  who  will  make  recommendations  to 
the  proposal  manager.  Any  agreed  to 
changes  and  proposal  improvements 
are  incorporated  into  the  proposal 
prior  to  initiating  the  formal 
review  process.  A  key  ingredient 
to  this  evaluation  is  a  "risk  assess¬ 
ment."  This  involves  taking  the 
RFP/SOW  requirements  and  evaluating 
the  probability  of  satisfying  these 
requirements  within  the  cost, 
schedule  and  terms  of  the  contem¬ 
plated  contract.  If  a  program  is 
deemed  to  be  a  "high  risk"  venture, 
additional  management  reviews  are 
required  to  obtain  approval 
to  propose.  One  of  the  techniques 
utilized  by  our  "red  teams"  is  to 
take  the  major  elements  of  the 


proposal  and  rank  them  in  order  of 
importance  which  is  felt  to  be 
consistent  with  the  way  the  buyer 
will  rank  them.  This  is  done  not 
only  with  the  technical  aspects  but 
also  cost  and  schedule.  A  deter¬ 
mination  is  made  as  to  whether  the 
proper  emphasis  has  been  placed  in 
the  manner  the  buyer's  priorities 
have  been  established.  Hopefully, 
utilizing  the  RFP  and  answers  to 
submitted  questions,  a  proper  align¬ 
ment  can  be  made.  If  it  turns  out 
an  area  has  been  over  (or  under) 
emphasized,  adjustment  must  be 
made. 

Jhe  problem  with  this  type  of 
evaluation  process  is  that  it  takes 
time,  and  time  is  normally  a  very 
rare  commodity.  To  turn  a  proposal 
around  in  45-60  days  when  there  are 
subcontractors  involved  is  difficult 
at  best.  By  the  time  the  technical 
approach  is  established  and  priced, 
there  is  little  or  no  time  left 
for  a  comprehensive  evaluation. 
Therefore,  it  is  necessary  to  get 
a  jump  on  the  RFP/SOW.  Very  often 
a  proposal  is  completed  or  near 
completion  before  RFP  is  issued. 

The  buyer's  needs  and  priorities 
are  determined  through  marketing 
efforts  and  hopefully  not  too  many 
changes  need  be  made  to  the 
technical  approach  already 
estab I i shed . 

This  early  start  is  essential,  as 
a  proposal  that  does  not  get  the 
necessary  review  before  it  is 
submitted  will  be  successful  only 
by  sheer  luck,  and  a  series  of 
poorly  organized  non-responsi ve 
proposals  may  damage  the  repu¬ 
tation  of  the  company. 

The  tendency  today  i s  to  try  and 
contract  on  a  fixed  price  basis, 
either  firm  fixed  price  or  some 
type  of  incentive  arrangement.  One 
of  the  decisions  a  seiler  must  make 
during  the  evaluation  process  is 
whether  the  SOW  and  type  of  contract 
are  compatible  and  whether  sufficient 
costs  and  time  have  been  provided 
for.  Very  often  ways  have  to  be 
sought  to  minimize  risk  while 
reducing  (or  at  least  maintaining) 
the  estimated  costs.  This  can  lead 
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II.  EVALUATION  PROCESS  (continued) 

to  alternate  proposals,  which  sellers 
normally  do  not  desire  preparing. 

Ml.  PROPOSAL  CHECKLIST 

A.  Technical  Approach 

This  section  should  provide  an 
analysis  of  the  problem,  a 
discussion  of  the  operational 
environment  and  an  accurate  and 
clear  technical  description  of  the 
proposed  system  and/or  hardware, 
including  drawings  or  sketches  of 
the  proposed  configuration.  The 
following  should  be  considered: 

1.  Is  there  a  clear  concise  state¬ 
ment  of  the  technical  regui ce¬ 
ments  which  the  proposal  ful¬ 
fills? 

2.  Is  the  technical  problem  as 
seen  by  the  customer  clearly 
delineated?  -  not  simply 
"parroted"  from  RFP?  Is  the 
proposal  responsive  to  the 
technical  requirements? 

3.  Does  the  proposal  convincingly 
show  a  depth  of  understanding 
of  the  problem? 

4.  Is  there  a  brief  dicussion  of 
alternate  solutions  which  were 
explored  and  rejected  and  the 
reason  for  their  rejections? 

5.  Is  there  a  discussion  of 
technical  approaches  to  be 
explored  and  why  the  company's 
approach  may  be  expected  to 
yield  the  desired  results? 

6.  Have  unrealistic  and  unreason¬ 
able  performance  requirements 
been  identified  and  alterna- 

t i ves  suggested? 

7.  Have  the  more  difficult  areas 
been  identified  and  detail  pro¬ 
vided  showing  how  performance 
requirements  never  before 
achieved  will  be  met? 


Have  excessive  costs  or 
1  ime  delays  ri'CuiriM)  do 
meet  certain  specific, 
rc'iu  i  rements  been  clearly 
pointed  out? 


9.  Is  there  a  statemeni  of 
major  technical  problems 
which  must  be  solved  with 
an  indication  as  to  the 
amount  of  effort  budgeted 
to  each? 


10.  Is  the  relation  of  proposed 
solution  to  the  broader 
over-all  system  with  which 
it  will  operate  shown? 

11.  Is  there  a  description  of 
the  hardware  which  the 
contractor  expects  to 

f urn i sh? 


12.  Does  the  proposal  state 
where  the  company  intends  to 
deviate  from  standard 
military  specifications? 

How  much?  Why? 

13.  Does  the  proposal  show  that 
proper  consideration  has 
been  given  in  the 
operational  concept  to 
serviceability  and  ease  of 
ma i ntenance? 

14.  Is  an  estimate  furnished  of 
maintenance  procedures  and 
schedule  showing  to  what 

u y+ent  special  test  or 
supp  't  equipment  will  be 
t  equl red? 

15.  If  new  components  must  be 
developed,  does  the  proposal 
explain  why  existing  ones 
cannot  be  used? 

16.  Are  unique  or  unusual  com¬ 
ponents  described  and 
justi fled? 

17.  Is  an  estimate  provided  as 
to  the  likelihood  of  the 
program  resulting  in  usable 
hardware? 
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PROPOSAL  CHECKLIST  (continued) 

7. 

Are  areas  of  technical  weakness 
identified  and  does  the  proposal 

■  .  - 

A.  Technical  Approach  (continued) 

show  how  this  will  be  compensated 
for  -  for  example  by  subcon- 

18.  Is  convincing  technical  sub- 

tracting  or  the  use  of  con- 

.  • 

stantiation  of  the  design 
proposed? 

su 1 tants? 

8. 

Does  the  proposal  clearly  indi- 

19.  Does  the  approach  avoid  over- 

cate  that  there  is  adequate 

engineering  and  over-soph  is- 

technical  space  and  facilities. 

tication? 

both  general  and  special  to 
perform  work  efficiently  and  on 

20.  Isa  manhour  estimate  (not 
costs)  included  in  the 

schedule? 

-  *.  “ 

technical  portion  of  the 

9. 

Does  the  proposal  outline  the 

propose  1 ? 

availability  of  the  facilities. 
Government-owned,  owned,  or 

B.  Techn i ca 1  Ability 

leased,  necessary  for  the 
specific  project,  for  research. 

This  Section  should  clearly  demon- 

development,  production  and 

strata  the  overall  technical 
competence  of  the  company  to 

test i ng? 

successfully  complete  the  specific 

10. 

Are  special  technical  facilities 

project  involved. 

(such  as  dust-free  laboratories, 
temperature  control  led  rooms. 

1.  Does  the  proposal  provide  con- 

data  processing  equipment. 

vincing  assurance  of  specific 

special  laboratory  equipment) 

technical  competence  for  this 

required  by  the  project  clearly 

project? 

spel led  out? 

2.  Does  the  proposal  give  specific 

11  . 

Is  it  clearly  indicated  that 

examples  of  similar  projects 

all  required  facilities  will  be 

*  .■  , 

successfully  completed? 

3.  Is  information  provided  as  to 

available  when  required  for  this 
project? 

the  relation  of  the  proposed 

12. 

Where  Government-furnished 

hardware  to  existing  or  previous 

equipment  is  required,  are 

programs  which  the  company  has 
done  for  other  customers, 

these  needs  clearly  justified? 

indicating  the  customer,  pro- 

13. 

Is  a  facility  plan  provided 

ject,  and  funds  already  spent? 

4.  Do  the  biographies  relate 

showing  layout,  tests,  dollar 
value  and  square  footage? 

specific  experience  of  personnel 

14. 

Where  tie-ins  with  subcon- 

# 

to  the  specific  needs  of  this 

tractors  are  proposed,  is 

project?  Has  extraneous  bio- 

specific  evidence  given  of  the 

-  ■'  'v  ■  -  ' 

graphical  information  been 

subs'  commitment  to  make 

el iminated? 

5.  Is  the  availability  of  specific 

technical  people  and  facilities 
available  when  required? 

l-t’ 

people  clearly  detailed  -  -  in 

C. 

Delivery  Requirements  and 

terms  of  manhours  for  both  full¬ 
time  and  part-time  people? 

— 

Schedu 1  i  nq 

\  .  .1 

Del 

ivery  is  most  important.  The 

6.  Since  the  customer  knows  that 

proposal  must  not  only  state  that  the 

the  same  personnel  are  used  for 

del 

ivery  schedule  will  be  met,  it 

different  proposals,  does  the 
proposal  show  a  depth  of 
qualified  personnel? 

must  show  how  it  will  be  met. 

-Ji 
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PROPOSAL  CHECKLIST  (continued) 


C.  Delivery  Requirements  and 
Schedu I i nq  (continued) 

1.  Does  the  propose,  provide 
convincing  assurance  that  the 
customer's  delivery  dates  will 
be  met  or  bettered? 

2.  Is  sufficient  detail  regarding 
master  scheduling,  programm¬ 
ing,  follow-up,  and  other  like 
functions  given  to  reinforce 
the  foregoing  assurance? 

5.  Where  subcontractors  and  major 
suppliers  are  involved,  are 
siifficient  safeguards  built 
into  proposed  scheduling 
system  to  insure  sub-schedule 
compliance  with  master  program? 

D.  Project  Direction  and  Manage¬ 
ment 

The  proposal  should  show  the 
company's  method  of  management. 

It  should  elaborate  on  organization 
personnel  manpower  controls.  It 
must  demonstrate  that  the  company 
has  an  understanding  of  the 
external  organization  relations 
with  the  Covernment  or  Prime  Con¬ 
tractor  and  with  subcontt'actors 
necessary  to  the  accomplishment  of 
the  project.  It  must  outline  the 
overall  management  concepts 
employed  by  the  company  and  the 
specific  type  of  management  that 
will  be  provided  for  tbe  proposal 
project. 

1.  Does  the  proposal  clearly 
demonstrate  an  understanding 
of  the  customer's  concern  with 
the  management  of  this  project? 

2.  Are  details  provided  on  cor¬ 
porate  experience,  facilities 
and  personnel? 

3.  Does  the  proposal  demonstrate 
that  top-level  management  will 
continue  a  high  level  of 
interest  and  assume  responsi¬ 
bility  for  successful  accom¬ 
plishment  of  the  program? 

4.  Is  evidence  given  of  manage¬ 
ment's  understanding  of  how 
the  specific  project  fits  into 
the  customer's  over-all  needs? 


b.  Are  detai  Is  provided  on  m.in.tiie- 
ment  objectives,  policic^'i, 

(lart  ic  i  put  i  on  ,  and  reliability 
concepts? 

6.  Does  the  proposal  show  how  the 
interest  of  the  company  in 
this  specific  project  ties  in 
with  the  company's  long-range 
plans  as  well  as  with  past 
exper i ence? 

7.  Does  the  propos.al  outline  the 
type  of  management  to  be  pro¬ 
vided  for  the  project. 

8.  Is  information  furnished  as  to 
the  type  freguency,  and  effec¬ 
tiveness  of  management  controls 
and  methods  for  corrective- 
action? 

9.  Do  the  manpower  buildup  charts 
clearly  explain  the  methods  of 
manpower  acquisition,  parti¬ 
cularly  skilled  manpower 

requ i rements? 

10.  Is  a  total  manpower  plan  and 
individual  plans  for  engineer¬ 
ing,  manufacturing  and  quality 
control  furnished? 

11.  Is  information  furnished  show¬ 
ing  how  the  present  project 
will  phase  in  with  current  and 
future  business? 

12.  Is  a  Make-or-Buy  Program  pro- 
V i ded? 

13.  Is  evidence  given  of  the  com¬ 
plete  support  of  the  subs' 
management  tor  an  arrangement 
wherein  the  company  would  be 
the  system  manager? 

E.  Quality  Assurance.  Quality 
Control  and  Reliability 

The  term  "Quality  Assurance"  covers 
all  the  actions  necessary  to  ade¬ 
quately  determine  that  product 
requirements  are  met.  "Quality 
Control"  is  the  system  and  manage¬ 
ment  function  by  which  the  Contraefor 
ascertains  and  controls  the  quality 
of  supplies  or  services.  "Reliability" 
is  the  ability  of  item  to  function 
without  failure.  The  proposal  sfiould 
carefully  delineate  the  company's 
programs  in  these  areas. 


PROPOSAL  CHECKLIST  (continued) 

E.  Quality  Assurance,  Quality 
Control  and  Reliability 
(conti nued) 

1.  Does  the  proposal  describe  the 
company's  quality  control  plan? 

2.  Is  it  clear  that  the  customer's 
quality  control  requirements 
will  be  achieved  by  the  company's 
quality  control  system? 

3.  Are  deviations  from  customer 
requirements  satisfactorily 
exp  I  a i ned? 

4.  Does  the  proposal  show  that 
customer  reliability  require¬ 
ments  can  be  achieved  by  the 
company's  concept  and  approach, 
including  a  specific  program 
for  meeting  or  surpassing  these 
requ i rements? 

5.  Is  it  clearly  shown  how  the 
reliability  organization  and 
project  responsibility  fit  into 
the  proposed  program? 


2.  Does  the  proposal  expain  the 
system  and  procedures  used  for 
schedule  planning  and  opera¬ 
tional  controls? 

3.  Does  the  proposal  provide  con¬ 
vincing  assurance  of  specific 
manufacturing  competence  in 
terms  of  this  project? 

4.  Does  the  proposal  clearly  indi¬ 
cate  that  the  company  has  ade¬ 
quate  manufacturing  space  and 
fac i I i ties? 

5.  Does  the  proposal  show  evidence 
of  an  effective  manufacturing 
control  system? 

6.  Does  the  proposal  state  that  all 
required  facilities  are  ava  i  I  able? 

7.  Does  the  proposal  provide  evi¬ 
dence  that  the  company  utilizes 
the  most  advanced  methods  in  its 
manufacturing  and  manufacturing 
support  areas? 

In  addition  to  the  above  sections, 
areas  such  as  price,  field  support, 
proposal  format,  would  be  covered 
i n  the  check  list. 


6,  Are  reliability  mon i tor i ng  po inis 
clearly  delineated  so  that 
customer  surveillance  may  be 
effectively  exercised? 

7,  Does  the  proposal  show  an 
understanding  of  reliability 
prediction  techniques  and  spell 
out  in  detail  how  predicted 
goa 1 s  w i I  I  be  met? 

8,  Is  creative  ingenuity  reflected 
in  the  proposal  by  pointing  out 
reliability  approaches  to  parti¬ 
cular  development  Phases? 

F.  Manufacturing 

The  proposal  should  show  the  company^ 
competence  to  manufacture  the  item. 
Some  information  in  this  field  is 
important  even  in  proposals  for 
research  and  development  which  may 
not  involve  any  quantity  production, 
since  the  buyer  must  usually  give 
consideration  to  an  plan  for  future 
production  quantities. 

1.  Does  the  proposal  describe  the 
company's  manufacturing  organi¬ 
zation  responsibilities? 


IV.  POST  AWARD  ANALYSIS 

The  evaluation  process  need  not  stop 
once  the  award  is  made.  In  addition 
to  any  buyer  initiated  de-briefing, 
it  may  be  advisable  to  conduct  an  in- 
house  evaluation  of  the  factors  that 
contribute  to  a  loss,  or  even  a  win. 
This  evaluation  can  be  very  beneficial 
in  future  proposal  activities.  Under¬ 
standing  "why"  the  buyer  reacted  to 
your  proposal  the  way  he  did  can  help 
in  making  the  next  proposal  more 
responsive  to  needs  of  the  buyer.  It 
is  equally  important  to  know  where 
you  did  well  in  order  that  you  can  do 
it  that  way  again.  This  evaluation 
can  be  used  to  determine  the  buyer's 
attitude  toward  your  firm  in  order 
that  strengths  can  be  exploited  and 
weaknesses  corrected. 

V.  SUMMARY 

Very  rarely,  if  ever,  does  a  company 
win  everything  they  b i d  on .  So  the 
key  thing  is  to  continually  improve 
your  win/loss  ratio.  One  of  the 


V.  SUMMARY  (continued) 


most  important  ways  of  doing  this 
is  to  understand  your  customer  and 
prepare  your  bids  so  they  are 
meaningful  to  him.  Internal 
evaluations  of  your  proposals  to 
insure  that  all  objectives  are 
being  met  is  mandatory  to  achieving 
a  goal  of  high  awards.  Having  the 
technical  capability  is  not  enough 
if  you  can ' t^ express  this  to  your 
customer  in  a  way  he  is  satisfied 
you  can  do  the  best  Job  for  him. 


Many  hours  and  dollars  can  be  spent 
developing  a  proposal  that  never 
get  your  message  across  and  thus  is 
not  selected.  Being  geared  to 
start  quickly  and  have  enough  time 
to  not  only  properly  evaluate  your 
product  but  also  make  improvements 
will  help  you  win  more  awards  and 
perform  at  a  profit.  Winning  is 
great,  but  not  if  the  risks  are  not 
consistent  with  the  price,  schedule, 
etc.  and  you  therefore  lose  money. 


Don't  be  afraid  to  ask  for  a 
de-briefing.  Most  buyers  are  very 
willing  to  give  you  some  insight 
into  how  your  proposal  was  evaluated. 
This  information  will  be  very  help¬ 
ful  the  next  time  around.  Keep 
records  of  what  you  thought  were  key 
concerns  of  the  buyer  and  whether 
you  were  right  or  not. 


m  ••L-d 


Tiir:  I’KorosAi,  camk 
l)K  SAM  SI  I.VKKHIIKC 
IIIICIIKS  AIKCHAI'T  CO 

I’KI'l^M  l  SI'. 

IT'S  NOT  MOW  cool)  A  CONTKACTOU  IS,  liUT 
KAI'III'.K  now  cool)  Till';  COVI'UNMI'.NT  IIVAI.IIA- 
I'OKS  i’i:iu:i;i VI'.  him  ro  hk  i'iiai'  is  oi'  con- 
si:()ui:nck  dukinc  i’koi’osai,  kvai.liai'ions. 

CIISTOMAKY  Coyi'UNMI'.NT/ 

ClIS'l'OMI'.K  INri';i<CIIANCKS 

O  Ml  DDKK'S  CONFKMI'.NCI'; 

0  KKSI’ONSI'.  TO  Ml  DDKKS'  C0NI'I'.KI';NCK 
0  DMArr  UI'I’/COMMI'.N'I'S 
0  I'KCIINICAI,  I’KI'Sl'NTA'nONS  MY 
CONTMACrORS 

0  "CASUAI,"  MI'.I'.TINCS  Wn  il  CON  I'RAC'I'OR 
MARKKT I  NC  RI'.I’RKSI'.NTA'I'  I  VP'.S 
0  COMMI'-.NTS  ON  FINAI,  RKI’ 

0  I’ROI'OSAI.  (WRIT'i'KN/ORAl.S) 

I’Rm^sTvi _WRriiiJi(; 

0  I’RCSKN'I'  Till'.  Ol'I'I'RINC  IN  Till',  Ml', ST 
l■OSSIMI,|•;  I.ICIIT 

0  AVOID  MI'.NTION  OK  YOUR  WKAKNI'.SSK S 
0  AI.I.UDI',  I'O  COMl’I'.rnOR'S  WIOAKNKSSKS 
0  CON'I'INUAI.I.Y  IN'I'KRWKAVI':  TKOS'K 
I'llKMI'.S  KNOWN  I'O  HK  IMPORTANT  TO 
TIM',  CUSTOMI'.R 

I’llASI',  II  I'HKMK,  DRII.I. 

1.  Wl',  IIAVK  A  UNIQUK  SOl.lITlON  TO  YOUR 
UNIOUK  RK.I^UIRI'.MK'.NTS. 

2,  DI'.DICA'I'KI)  I'OTAI,  ORGANIZATION  -  NEW 
FOR  THIS  ,10 M. 

J.  UNIOIH',  ORCANIZATIONAI,  STRUCTURK  - 
"CKAI)l,l',-ro-CRAVK,-"  MAN  AG  K, MEN  r  CONI' I NU  I 'PY . 
A.  ADVANCI'.I)  UNDERS'l'ANDING  OK  JOM  DUE  I'O 
SICNIKICAN'I'  PRIOR  I  NVESTMEN'I'S . 

5.  I'.XIS'l'ING  PROGRAM  X  REKINIIS  OUR  ■I'ECIl- 
NOEOCICAI,  UNDIIRS'I'ANDINC  FOR  PROGRAM  Y. 
b.  riCAMED  WITH  ESTAME I  SHED  SISTER 

commi:rciai,  division  io  offer  credimi.e 

"IN-HOUSE"  EOW  COST  OF  MO'I'H. 

7,  UTII.I/.ING  INrEGRAI,  EI.ECTRONICS  IN  A 
PREVIOUSI.Y  AI.I.-MI'.CHANICAI,  DEVICE  SIMPI.I- 
Fll'.l)  CONSTRUCTION  WIIIEF,  YIEEDING  SUPERIOR 
I’l-RI'ORMANCi:. 

H.  EOW  RISK  DEVISED  FROM  OPERATING 
MRKADMOARDS. 

<7.  KEY  PAYEOAD  I'.EEMENT  HAS  FEOWN 
SUCCTSSFIIEEY  THF.RTTIY  REDUCING  RISK. 

10.  PR0VT;N  PI'.RFORMANCI',  -  our  products 
HAVE  AEREADY  FEOWN  TO  'I'lll',  MOON  AND 


RETURNED  AND  ARE  NOW  KEYING  MY  .lUIM  TER. 

11.  OUR  Tl'CHNICAE  EXPERT  I  SI',  WIEE  PROVIDE 
SUPERIOR  DESIGN  AND  PIIRFORMANCT,  AND  Wl', 

ARK  'TEAMING  WI'TII  OUR  SIS'TTIR  CORPORATT', 
DIVISION  WITH  IIS  YI'.ARS  OF  PROVI'N  MANU- 
FAC'TURINC  I'.XPIOTI  I'.NCT:  TO  INSURI'.  DI:E  I VIIRY 
OF  'THE  PRODUCTION  PRODUCT  ON  SCHI'DUEF, 

AND  AT  EOWES'T  POSSIMET',  COST. 

12.  INDEPI'.NDT'.N'T  ACCOUN'T  I  NC  STIGMT'.N  T 

formi;d  to  taieor  ovi;rhi;ad  and  v.bA  to 

lINKJUE  ASPECTS  OF  BUSINESS, 
n.  OUR  COMPANY  IS  UNICJUF.EY  SUITED  TO 
MU  I  ED  THE  FtlEE  MISSION  SIMUEA'TOR 
MF.CAUSK  WE  MUIE'T  THE  AIRPEANE. 
lA.  WE  HAVE  MAXIMIZED  PERFORMANCE 
WIIIEF,  ACHIEVING  A  MAEANCE  BETWEEN  EOW 
INITIAE  ACQUISITION  AND  EIFE  CYCEE 
COSTS . 

15.  OUR  EXHAUSTIVE  TRADE  STUDIES  INSURE 
EOW  EIFE  CYCEE  COSTS  WITH  EXISTING 
'TECIINOEOGY  ITEMS  MATCHING  AEE  REQUIRE¬ 
MENTS. 

H>.  OUR  AIRPEANE  OFFERS  MAX  OPERA'TIONAE 
UTIEI'TY  WI'TII  A  MINIMUM  CREW  COMPEF.MENT- 
EOWER  ECC. 

17.  S'TA'TE  OF  THE  ART  TECIINOEOGY 
INSURES  U'TIEl'TY  AND  GROWTH  WI'.EE  INTO 
1990  TIMEFRAME, 

18.  'Tin;  AVIONICS  SUITE  IS  DESIGNED  FOR 
EASE  OF  EXPANSION  AND  RE-PROGRAMMING 
wrni  MINIMUM  AIRCRAFT  MODS. 

19.  OUR  AIRCRAF'T  PERI'ORMANCE  HAS  MEEN 
DEMONSTRATED  SUPERIOR  TO  AEE  OFFERORS. 

20.  WE  USE  OUR  OWN  MANU FACTIIRED  FII;ED- 
PROVEN  PARTS  TO  SAVE  DEVEEOPMENT  COSTS 
THROUGH  S  TANDARD  I  ZA'T  I  ON  . 

21.  WE  HAVE  MEEN  USING  CORPORATE  FUNDS 
TO  DEVEEOP  THIS  TECIINOEOGY  FOR  YEARS, 

AND  AEE  THIS  EXPERIENCE  IS  AVAIEAMEE  AT 
NO  COST. 

22.  WE  HAVE  BEEN  MANUFACTURING  MORE 
THAN  8000  DIFFERIIN'T  PRODUCTS  FOR  MORE 
THAN  50  YEARS,  MANY  OF  WHICH  REIRURE 
SIMIEAR  TECHNIQUES. 

23.  WE  ARE  AMONG  TIM',  PIONEERS  IN 
CRYOGENICS  RESEARCH  AND  DEVEEOPMENT, 

AND  UNDERSTAND  'THE  REQUIREMENTS 
COMPEETEEY. 

24.  WE  HAVE  BUIE'T  AND  TESTED  A  DEMON¬ 
STRATION  UNIT  WITH  CORPORATE  FUNDING 
AND  ARE  COMPEETEEY  COMMITTED  TO  THE 
TECIINOEOGY. 

25.  tut:  proposed  system  is  SIMIEAR  TO 
THOSE  AEREADY  BEING  MANUFACTURED  AT 
ONE  OF  OUR  DIVISIONS. 

26.  WE  HAVE  Till',  REQUIRED  PI'RSONNEE  IN 
PEACE,  AVAIEAMEE,  AND  UP  TO  SPEED. 

27.  WE  ARE  A  RECOGNIZED  EKADER  IN  THIS 
KiEED  AS  di:mons'tra'ti;d  by  hundreds  of 
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KKLATKI)  CONTUAC'I'S,  WOKl.DW  1  DK . 

28.  WK  IIAVI':  AN  OUTSTANI) I NC  KKCOKI)  OK 

mana(;i;mi;nt  on  similar  kroorams  in  tkrms 

OK  TKClINOl.OCY  AND  1)01, 1.AR  VAI.UK. 

29.  wl;  iiavk  donl;  i;xTi;NSivi';  inticrnal 
STUD  IKS  and  UNDI:rSTANI)  'I'IIL;  customkr 
NKI'.DS . 

jO.  our  COMI’ANY  mas  I’ROVKN  RI'CORD  ok 
ON-TIMI'.  DI'I.IVI'.RII'.S,  WII'IIIN  COST. 

)l.  OUR  COMI’ANY  MAS  I’ROVKN  RKCORD  OK' 

99Z  RI'.KIAIUKITY  IN  MICROWAVK,  RKI.AYS. 

)2.  OUR  COMI’ANY  DKVI'.Lni’l';!)  'l'lll■•.  I’ROTOrYI’i;. 
(}.  I’l'.RSONNKK  WHO  dkvkkoi’i;d  and  dkmon- 
strati;d  men  l’l■:RKORMANCl■:  ti'.ciinokocy 

AVAIKA8I.I',  I'OR  TIMS  I’ROCRAM. 
lA.  Wl'l'll  OUR  Tl'ClINICAK  AND  MAN ACI'MI'.N'I' 
ORCANI/.ATION  IN-PI.ACi:,  NO  I’ROCRAM  TIMK, 

OR  DOKKAR  RI'.SOURCKS  WILL  UK  WAS'IKD 
RI'.CRUITINC  AND  TRAIN  INC  AN  ORCANI/.ATION. 
J').  OUR  TI'.CIINICAL  I’KRKORMANCK  IS  DOCU- 
MI'lNTi:!)  IN  YOU  UR  RI'ICORDS  WHICH  SHOW  NO 
KAIKURI’IS  IN  THK  CAST  (X)  AN-YY  AND  (Y) 
AI’S-RR  UNITS. 

K).  ALL  PRODUCTION  l•;(}U  I  I’MI'NT  RITJUIRKD 
TO  I  NIT  I  ATI'.,  HIM  LI)  TO  AND  MAINTAIN  YOUR 

Ri'.ijuiRKD  di;li vi;ry  ratl;  is  in  plack  and 

RKADY. 

17.  SI  NCI'.  YOUR  MANACI'.Ml'N'l'  SYSTI-'.M 
INDICA'I'OR  SYS'I'I'.M  was  INITI  AI'I':D  all  msi's 
IIAVI’,  UKI'.N  "CRI'.KN". 

38.  I'lll';  OI’I'IRAI'INC  I’UO'I'OTYI’I';  Ol'  OUR 
I’ROl’OSI'.D  DI'.SICN  HAS  ALRI'ADY  SI'I'.N  (X) 

I’.I'NCH  HOURS  AND  (Y)  I'.NV  I  RONMI'.N'I'AL  TKST 
HOURS  WI'I'HOU'I'  A  KAII.lIRi;. 

39.  IN  OVL'.R  A  Dl'CADI''.  OL'  Di;S  I  CN  I  NC  . 
DI'lVIM.Ol’INC  AND  I’RODIUMNC  X'S,  I'.VKRY 
DKLIVKRY  HAS  IIIM'.N  ON  OR  AHI'.AD  OL'  I'llK 
ORJCJNAL  CONTRACT  SCIH'.DUI.T;. 

/(O*.’ '  t'iH';  i’roi’osi;d  uasi'.lint:  di'.sicn  has 
iH'.i'N  n  KRA'i'i'.i)  thri;t;  timt.s  to  i  di:nti  i'y 

ALL  COST  DRIVKRS  AND  TIIT'IR  IMPACTS. 

41.  IN  THIS  DI'.SICN  PROCURTIMI'.N'T  -  OUR 
DKSICN  IS  ALRI'.ADY  COMPLI';'TI',D . 

42.  OUR  TKS'T  KACILI'TIKS  SI'.T'.M  TO  IIAVK 

iiT'.l'iN  di:sicnt;d  with  your  hardwart;  in 

MIND. 

43.  OUR  TOP  MANACKMKN’T  PTiRSONALLY  ART', 

(:onct:rni;i)  wi  th  and  commi'T'ti'.d  to  this 
procuri;mi:n't. 

44.  our  cost  CON'TROL  SYS'TI'.M  PROVIDT'.S 
YOU  A  "DAIKY  LOOK  THRU  THT;  WINDOW  01'  THK 
OVK.N". 

45.  MOST  KXI’TIRI  KNCK  IN  I'IT;LD  -  (SHOW 
PHOTO  OK  ICARI.Y  PRODUCT). 

46.  S'TRONC  TRACT  RKCORD  KOR  IN-THK- 
KIKLD  SUPPORT  DURINC  I  N'TICCRA'T  I  ON  AND 
KLICH'T  TKS'T. 


47.  incrt'.asinc  kmpiiasis  on  human 
kn(mnkt:rinc  and  tut:  man-machint;  intt.r- 

KACK  HAS  RI:DUCI'.D  OPTIRATINC  COMPI.I'.XI'TY. 

48.  CARKKULKY  THOUCITT-OU'T  WORK-AROUND 
S'TRKSSKS  T;ARLY  PR08LT;M  DT'.'TKC'TION. 

49.  WK  CAN  lil’S'T  I  N'TKCRA'TT',  Till'.  MUK'TI- 
1)  I  SC  It’Ll  NARY  TI'.CIINOLOCI  T'.S  RK()U  I  RT;D. 

50.  LARCK,  RKMD,  L  I  CH'TWK  I  CUT  S'TRUCT- 

URKS,  nt'.i'.dt;d  to  ptirkorm  this  mission 

ARK  AVAII.AIILK  IN  OUR  COMPOS  I  TK  MA'TIIRIAKS 
TKClINOl.OCY. 

51.  OUR  SUUCON'TRAC’TOR  TT'.AM  HAS  UIM'.N 
CARKKULKY  SI'.LT;C'TT;D  to  add  ItRI'.AD'TH  AND 
Di;p'Tii. 

52.  wt;  art;  currt;n'tly  pt;rkorminc  similar 

WORK  KOR  tht:rt;i5Y  rkdiicinc  risk. 

53.  TH  I  S  PROCr'aM  T'.NHANCTiS  OUR  CAPAIM  L- 
I'TY  IN  AN  ALRI'.ADY  PROVKN  PRODUCT  LINT'.. 

54.  WK  HAVT:  IITIT'.N  VAI .  I  DA'TT'.D  ACAINS'T 
esese  SI  NCI.  1969. 

55.  WK  UNDTIRS'TAND  THT',  I’ROIII .T'.MS  TO  IIT; 
SOI.VKI)  AND  THK  MIM'HODS  RT;()U  I  RT'.D  TO 
MINIMl:'.i:  THT.  IR  IMPACT. 

56.  AD  tut;  I nciimht'.N'T,  wt;  okt't;r  cost 
SAVINCS  RT;ALI/T'.D  T'ROM  an  on-iioard, 
TRAINKI),  AND  DT'.D  I  CA'TT'.D  T'UNC'T  I  ONAI . 
ORCANIZA'TION. 

57.  OUR  SIIPPKIT'.RS  AND  SUDS  DISTRI8UTT'. 
YOUR  INVT'.S'TMT'.N'T  COS  TS  TO  S  TA  TI'S  AND 

. T'ORI'.ICN  (lOUN'TR  I  T'.S. 

58.  '  OUR  CONST.RVA'TI  VT;  APPROACH  UST'.S 
I’ROVKN  dt;sicn  t'or  iipcradt;  ratht;r  than 
•  IIIMPINC  TO  NT'.W  DT'.VT'.L0PMT;N'T. 

59.  wt;  havt;  lowt;rt;d  t;i)u i pmt;n't  costs  iiy 
DKSICN  INC  SYS'TT;M  I  N  TT'.RT'ACT.S  optimi/.t; 

USK  OK  CT'T.. 

60.  T■KT;XI  111  1,1  TY  OT'  ACCOMMODA  I  T.  SYS'TTIM 

in'tt'.rkact;  mods  is  providt.d  wi  th  "our 
COMPANY"  t'irmwart;  con  trollt;r  . 

61.  A  I.OWT'.R  COST  SOLUTION  TO  HIT;  PROIll.TIM 
OK  CONTI  NOUS  ON-LINT;  T'AULT  DT;  TT'.C'T  I  ON  IS 

pro(;t:sst'.r  ariiiu  tticturt;. 

62.  OUR  DKSICN  PROVIDTIS  DTIS  TR  I  liUTT;D 
SOLID  STA'TT.  M I  CROPROCT'.SSOR  CONTROK  OT' 

X/./.  SYS'TIIM  WI  TH  ALL  Till',  RT',I)U  I  RT'.D  T' I  RM 
WART.  AND  SOT'TWART,. 

6  1.  OUR  CURRT;N'T  IRM)  HAS  CIVI.N  US  NKW, 

UNiijuK  tt:(;hnical  approac.iit.s. 

64.  OUR  RM)  APPROAC.H  IS  N0VT;L  AND 

PRovisiNc  wht;n  linkt'.d  up  wi  th  tt;st 

KACILI'TIKS  OT'  OUR  COMMI  TTT;!)  SU ll(;( )N  I  R A(  1- 
TOR. 

65.  "OUR  unii)ut;  systt;m  accuracy  on  I.ONC 
T'KICITTS  -  down  to  I.TISS  THAN  2  MM 
"i’rovt;n!" 

66.  sicnit'I(;an(;t',  ot'  our  worldwidt; 
st;rvi(;t;  loiiations  t'or  lt.vkks  ii  and  mi 


67.  OUTSTANDING  ACCURACY  ACHIEVED  BY 
OUR  INFLIGHT  INSPECTION  SYSTEM. 

68.  INFLIGHT  REFUELING  ACCOMPLISHED 
USING  PRECISION  AIRBORNE  NAV  SYSTEM. 

69.  WE  ENCOURAGE  THE  AIR  FORCE  TO 
LOCATE  PERSONNEL  AT  OUR  PLANT  TO  WORK 
WITH  US  AND  MONITOR  OUR  PERFORMANCE. 

70.  THIS  PROGRAM  REQUIRES  THE  SAME 
SKILLS,  TECHNOLOGY  AND  EXPERTISE  AS 
PROGRAM  X  WHICH  WE  DELIVERED  AHEAD  OF 
SCHEDULE. 

71.  WE  WILL  RETAIN  THE  VITAL  ELEMENTS 
OF  THE  SYSTEMS  JOB  IN-HOUSE  PROVIDING 
US  THE  NECESSARY  CONTROL  TO  MEET  COST 
AND  SCHEDULE  REQUIREMENTS. 

72.  OUR  APPROACH  PUTS  MORE  FUNCTIONS 
INTO  HARDWARE  TO  REDUCE  THE  RISK 
ASSOCIATED  WITH  COMPLEX  SOFTWARE. 

73.  OUR  SOFTWARE  SUB-CONTRACTOR  HAS 
MORE  SUCCESSFUL  EXPERIENCE  ON  THIS 
TYPE  OF  SYSTEM  THAN  ANY  OTHER  COMPANY. 

7A.  WE  HAVE  CO-LOCATED  THE  HARDWARE 
AND  SOFTWARE  DESIGNERS  IN  ORDER  TO 
MINIMIZE  RISK. 

75.  WE  ARE  LOCATED  RIGHT  OUTSIDE  YOUR 
GATES,  (OUR  COMPETITORS  ARE  ACROSS  THE 
COUNTRY). 

76.  THIS  IS  TSe  same  TEAM  THAT  GAVE 

YOU  THE  HIGHLY  SUCCESSFUL _ 

PROGRAM. 

77.  SINCE  THE  L-2  IS  SMALLER  THAN  THE 
L-1,  IT  IS  THE  ONLY  AIRCARFT  EVER 
BUILT  WITH  PROVEN  GROWTH  CAPABILITY. 

78.  AS  EVIDENCE  OF  OUR  LOW  RISK 
APPROACH,  WE  HAVE  SELECTED  A  SOFTWARE 
SUBCONTRACTOR  THAT  HAS  SUCCESSFULLY 
DEVELOPED  SEVERAL  TURNKEY  SYSTEMS  ON 
THE  NEW  COMPUTER. 

79.  WE  OFFER  THE  STABILITY  OF  A  LARGE 
COMPANY  WITH  THE  MANAGEMENT  VIGOR  AND 
EFFICIENCY  OF  A  SMALL  COMPANY. 

80.  AS  EVIDENT  BY  DESIGN  DETAILS 
PRESENTED  IN  THIS  PROPOSAL,  WE  ARE 
READY  TO  IMMEDIATELY  BEGIN  IMPLEMENTA¬ 
TION. 

81.  WE  HAVE  DEVELOPED  ENGINEERING 
MODELS  AS  "UP  FRONT"  PROOF  OF  DESIGN. 

82.  WE  HAVE  INITIATED  DESIGN  OF  THE 
SYSTEM  ON  OUR  OWN  RESOURCES. 

83.  ANOTHER  CONTRACTOR  WOULD  BE  FORCED 
TO  EXPEND  ENORMOUS  EXPENSE  TO  REPLICATE 
THE  FACILITIES  AND  EXPERIENCE  THAT  WE 
POSSESS  TO  SOLVE  THE  PROBLEM. 

8A.  BY  VIRTUE  OF  OUR  EXISTING  HIGHLY 
TRAINED  TEAM  OF  AVIONICS  ENGINEERS  AND 
TECHNICIANS,  WE  CAN  DO  THIS  MODIFICATION 
IN  HALF  THE  TIME  AND  TWICE  THE  PRECISION. 


85.  WE  ALREADY  HAVE  AN  INTEGRATED 
APPROACH  TO  LOGISTICS  SUPPORT  AND  HAVE 
SINGLE  POINT  OF  CONTROL  FOR  ALL 
ASPECTS  OF  INITIAL  AND  RECURRING 
VEHICLE  SUPPORT. 

86.  OUR  DESIGN  MEETS  OR  EXCEEDS  ALL 
RFP  REQUIREMENTS  WITHOUT  SIGNIFICANT 
TECHNICAL  RISKS. 

87.  SINCE  ALL  HARDWARE  HAS  BEEN  BUILT 
AND  TESTED  SUCCESSFULLY,  THE  WEIGHT 
ESTIMATES  ARE  REALISTIC. 

88.  WE  HAVE  THE  EXPERIENCE  TO  UNDER¬ 
STAND  THE  PROBLEM,  THE  TECHNOLOGY  TO 
SOLVE  IT,  THE  CAPACITY  TO  PRODUCE  IT, 
AND  THE  MANAGEMENT  TO  CONTROL  IT. 

89.  ENVIRONMENTAL  IMPACT  ANALYSES  ARE 
ALREADY  COMPLETE. 

90.  WE'RE  USING  "OFF-THE-SHELF"  - 
THIS  HEAD  START  REDUCES  YOUR  RISKS  AND 
YOUR  COSTS. 

91.  WE  HAVE  JOINED  WITH  NATIONALLY 
REPUTED  EXPERTS  FOR  THIS  JOB.  OUR 
SUCCESS  IS  ASSURED. 

92.  WILL  COST  SHARE. 

PROBLEM 

HOW  CAN  A  GOVERNMENT  EVALUATION  TEAM 
ASSURE  ITSELF  THAT  ITS  PERCEPTION  OF  A 
CONTRACTOR  IS  AN  ACCURATE  IMAGE  OF  THAT 
CONTRACTOR'S  CAPABILITIES  RATHER  THAN 
AN  ILLUSION  PAINTED  BY  THE  CONTRACTOR'S 
PROPOSAL? 

PROPOSED  SOLUTION 

A  TEAM  OF  SPO  PERSONNEL  WITH  ROTATING 
ASSIGNMENTS  AT  EACH  OF  THE  BIDDER'S 
FACILITIES  —  STARTING  PERHAPS  3 
MONTHS  PRIOR  TO  FINAL  RFP  —  TO  DEVELOP 
IN-DEPTH  KNOWLEDGE  ABOUT  EACH  CONTRAC¬ 
TOR'S  OFFERING. 

FUNCTIONS  OF  THE 
ROTATING  TEAM 

0  LEARN  ABOUT  EACH  CONTRACTOR'S 
DESIGN:  ITS  STRENGTHS  AND  ITS  WEAK¬ 
NESSES 

0  LEARN  ABOUT  EACH  CONTRACTOR'S 
PROPOSED  PERSONNEL  AND  FACILITY  AVAIL¬ 
ABILITY 

0  DURING  THE  PROPOSAL  EVALUATION, 
FUNCTION  AS  PART  OF  THE  EVALUATION 
TEAM 

0  HELP  PREPARE  QUESTIONS  ADDRESS¬ 
ING  APPARENT  WEAKNESSES  IN  DESIGN  OR 
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MANAGEMENT  OFFERING 


PROBLEMS  WITH  THE 
"PROPOSED  SOLUTION" 


GOVERNMENT 


0  NEED  MY  PEOPLE  AT  HOME  TO 
GENERATE  RFP  SPECS,  SOW,  ETC. 

0  CAN’T  AFFORD  3  PEOPLE  TDY  FOR 
6  MONTHS 

0  PLACES  TOO  MUCH  OF  EVALUATION 
RESPONSIBILITY  ON  ONLY  3  PEOPLE 

CONTRACTORS 


0  LOVE  TO  HAVE  YOU  BUT  NEED  TIME  TO 
COLLECT  MY  OWN  THOUGHTS 
0  ROTATING  ASSIGNMENTS  BETWEEN 

CONTRACTORS  MAY  PRODUCE  "TECHNICAL 
LEVELING" 
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PROPOSAL  EVALUATION 
JOSEPH  W.  KRUEGER 
STAFF  SOURCE  SELECTION  OFFICER 
AIR  FORCE  SPACE  DIVISION,  LAAFS 

Source  selection,  or  more  correctly  for 
todays  discussion,  proposal  evaluation  is 
a  simple,  uncomplicated  and  logical  pro¬ 
cess.  Only  people  complicate  the  process. 

Like  any  contest  or  sport,  all  are  con¬ 
ducted  according  to  a  rule  book,  (Chart  U 
In  the  case  of  proposal  evaluation,  the 
Department  of  Defense  some  years  ago 
issued  a  directive  4105.62  to  form  the 
basis  for  a  coninon  approach  to  negotiated 
competitive  acquisitions  of  major  propor¬ 
tions.  The  Air  Force  implemented  this 
direction  with  a  regulation,  70-15  is  the 
number.  It  contains  basic  guidance  and 
requirements  and  paraphrases  public  law 
and  the  Defense  Acquisition  Regulation 
(DAR).  Systems  Command  and  the  Space 
Division  also  implemented  this  regulation 
as  well  as  producing  guidance  and  policy 
for  the  structuring  and  content  of  the 
Request  for  Proposal  which  initiates  the 
competitions  in  which  industry  engages. 
Systems  Command  has  a  pamphlet  AFSCP  70-4 
and  the  Space  Division  has  a  regulation 
number  SDR  70-2  which  provide  policy  and 
guidance  for  uniformity  in  RFP  structuring. 

While  some  new  wrinkles  in  the  overall 
process  have  been  introduced  by  the  Systems 
Command  and  the  DAR  set  forth  a  new  pro¬ 
cedure  for  conducting  the  process  known 
as  the  Four  Step  Procedure,  (Chart  2). 

The  basic  technique  or  methodology  of 
evaluation  has  really  remained  unchanged 
in  the  20  years  I  have  served  the  Govern¬ 
ment.  We  have,  however,  added  other 
elements  of  consideration,  a  little  dif¬ 
ferently,  such  as  past  perrormance,  RFP 
tailoring,  environmental  awareness.  Some 
things  are  more  important  today  than  they 
were  years  ago  in  making  the  final  deter¬ 
mination  of  a  winner  but  the  basic  evalua¬ 
tion  is  still  the  same. 

I  also  should  mention  that  at  the  Space 
Division  over  a  period  of  a  number  of 
years,  (Chart  3),  the  offeror  proposing 
the  low  dollar  figure  for  an  acquisition 
was  coincidentally  the  winner  of  the  com¬ 
petition  less  than  one-third  of  the  time. 
This  is  irrespective  of  contract  type. 


The  part  of  the  RFP  which  sets  the  stage 
for  the  proposals,  is  Part  IV,  Section  L, 
which  at  Space  Division  is  subdivided 
into  an  L-1  and  an  L-2.  The  latter  is 
the  proposal  preparation  instructions, 
and  the  former  contains  important  notices 
about  the  proposal  effort.  This  is  where 
you  are  told  what  elements  of  the  acqui¬ 
sition  are  to  be  discussed,  how  the  text 
should  be  arranged  in  volumes,  the  limi¬ 
tation  of  type  size  and  foldouts,  and  the 
number  of  pages  you  are  expected  to  use 
for  each  volume,  (Chart  4).  In  the  case 
of  mission  assurance  or  sometimes  called 
product  assurance,  we  address,  in  addi¬ 
tion  to  the  risks  involved,  such  topics 
as  System  Safety,  reliability,  maintain¬ 
ability,  quality  assurance  and  produce- 
ability,  to  name  a  few,  (Chart  5).  Any 
failure  on  your  part  to  seriously  and 
conscientiously  address  what  is  requested 
in  this  or  other  sections  of  the  instruc¬ 
tions  can  spell  failure  to  win  a  competi¬ 
tion.  Generally  it  is  a  combination  of 
many  elements  which  spells  defeat,  but  in 
a  photo  finish  type  of  situation,  just 
one  element  may  be  the  straw  that  breaks 
the  camel ' s  back. 

The  next  section  or  Section  M  of  the  RFP 
is  where  wc  set  forth  the  ground  rules 
for  interpretation  of  your  response. 

First  of  all  we  tell  you  what 

part  of  your  proposal  is  the  most  impor¬ 
tant,  such  as  technical,  or  whether  all 
<najor  areas  are  considered  equally  im¬ 
portant.  These  are  called  areas  of  con¬ 
sideration,  (Chart  6).  We  next  tell  you 
what  the  yardsticks  are  for  measuring 
your  response  to  a  specific  area.  These 
are  called  criteria.  Some  coiTinon  criteria 
are  understanding  of  the  requirements, 
soundness  of  approach,  or  risk  minimiza¬ 
tion  and  most  recently,  since  1  Nov  79, 
past  performance,  (Chart  7).  The  relative 
importance  of  these  are  narratively  set 
forth.  Section  M  then  lists  in  outline 
format  the  items  of  consideration  and  the 
subdivision,  thereof,  called  factors  of 
consideration.  The  order  of  precedence 
of  these  is  also  specified,  (Chart  8). 

The  outline  headings  or  items  and  factors 
relate  directly  to  what  you  were  to 
supply  in  response  to  Section  L-2,  as  well 
as  to  the  tasks  of  the  work  statement  and 
the  work  breakdown  structure,  and  the 
contract  data  requirements  list,  or  CDRL. 
Each  area  of  consideration  gets  similar 
treatment  with  the  cost  proposal  not 
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being  rated  or  scored  but  a  very  vital 
part  of  the  total  package. 

With  both  of  us  reading  from  the  same 
rule  book,  we  are  in  a  position  to  not 
only  evaluate  your  proposal,  but  to  pre¬ 
sent  facts  and  figures  to  a  decision 
maker  as  to  how  well  you  responded  to  the 
various  elements  in  terms  of  the  criteria 
included  in  Section  M,  (Chart  9).  How 
sound  does  your  approach  to  various 
elements  appear  to  be?  flow  well  have  you 
recognized  and  faced  up  to  the  risk 
characteristics  of  approaches  taken,  and 
what  is  your  track  record  for  successfully 
accomplishing  similar  elements  on  other 
Government  contracts? 

In  proceeding  logically  to  accumulate  and 
summarize  the  data  from  your  proposals, 
an  evaluator,  (|ualifiod  to  assess  speciffc 
elements,  renders  several  pieces  of  docu¬ 
mentation  for  each  item  or  factor  eval¬ 
uated,  (Chart  10).  This  is  followed  by 
providing  a  narrative  which  relates  to 
the  stron(j  and  weak  points  previously 
identified,  (Chart  II). 

Now,  particularly  for  the  weak  points,  or 
for  other  reasons,  the  evaluator  will 
execute  a  clarification  re<iuest  or  CR 
which  later  will  be  edited  and  provided 
the  offeror.  Some  elements  or  presenta¬ 
tions  are  not  clear,  blocking  a  full 
appreciation  of  what  was  presented.  In 
other  cases  perhaps  some  backup  data  was 
not  provided  to  substantiate  an  assertion , 
or  the  basis  for  a  conclusion  .is  not 
stated.  Such  is  the  nature  of  the  CRs. 

Another  document  of  importance  to  the 
offeror  which  the  evaluator  completes  is 
a  deficiency  notice  or  I)N,  (Chart  12). 

By  definition,  this  is  a  portion  of  a 
proposal  which  fails  to  meet  the  require¬ 
ments  of  the  soloci tation.  In  the  case 
of  the  four-step  procedure,  these  defi¬ 
ciencies  will  only  be  revealed  to  the 
offeror  selected  for  negotiation  of  a 
definitive  contract.  Since  only  clarifi¬ 
cation  requests  are  submitted  to  an 
offeror  in  the  four-step  procedure.  It 
is  incumbent  on  the  offeror  under  this 
restrictive  procedure  to  initially  pre¬ 
sent  his  best  and  most  comprehensive 
proposal.  Deficiencies  are  the  serious 
weaknesses.  In  the  conventional  selec¬ 
tion  process,  the  DNs  are  identified  to 


each  offeror  and  the  offeror  has  an 
opportunity  to  correct  them  prior  to  the 
selection  decision. 

In  addition,  evaluator  personnel  cooper¬ 
ate  with  cost  analysis  personnel  to 
develop  risk  assessments  for  specific 
elements  of  the  requirement,  (Chart  13). 
These  form  the  basis  for  the  Government's 
development  of  the  most  probable  cost 
for  each  rfferor's  approach. 

In  addition  to  the  evaluators.  Govern¬ 
ment  advisors  and  selected  consultants 
are  utilized,  assuring  a  coiiinon  under¬ 
standing  of  the  content  of  each  proposal. 
All  the  documentation  generated  by  the 
evaluators  is  reviewed  by  team  chiefs, 
summarized,  and  important  CRs  and  DNs, 
stroru)  points  and  weak  points  elevated 
to  the  item  and  arc'a  lev('l,  (Cliart  M). 

A  final  summarization  is  pr(’senl('d  to 
the  decision  maker,  utilizini)  color 
cod(Hi  audio  visual  aid  charts,  along 
with  a  na rrativ(*  explaining  strong  and 
weak  points,  CRs  and  DNs,  all  ri’f  lectinc) 
the  responsiveness  of  the  oTTeror  in  tfu' 
terms  of  the  ori()inal  criteria  con tai ru'd 
in  the  solicitation,  (Chart  lli). 

Offeror  responsiveness  to  the  mission 
assurance  reepi i rements  are  highl  i()ht('d 
for  the  decision  maker  in  th(.'  l)ri('Tin()s 
provided.  The  briefitnis  arc*  based  on 
the  detailed  source  board  rc'port,  which 
contains  a  comprehensive  assessment  of 
each  offeror's  response,  (Chart  16). 

Some  thirty  to  forty-five  days  are  con¬ 
sumed  by  evaluator  teams  in  completing 
an  initial  assessment  of  the  three  to 
five  proposals  usually  received.  This 
includes  review  of  initial  evaluatons, 
challenges  of  findings,  liaison  with 
advisors  and  consultants,  resolution  of 
differences  of  opinion,  refinement  of 
narratives  and  suimiar i zations  for  brief¬ 
ing  presentations,  (Chart  17). 

Plans  submitted  for  management-and- 
conduct  of  quality  assurance  receive  the 
same  treatment  as  the  technical  a|»proach 
to  satisfying  a  reejui remen t.  Any  mission 
assurance  approach  providc’d  by  an  offc'ror 
which  essentially  states  that  the*  re¬ 
quirements  of  a  MIL  SPEC  will  be*  sat  is  i- 
fied,  won't  create  much  excitement.  I  or 
a  manned  orbit inc)  flight,  a  missile  or  a 


spacecraft,  we  want  to  know  what  unusual 
efforts,  reviews  or  analyses  will  be  con¬ 
ducted  to  enhance  the  assurance  require¬ 
ment.  Naturally  each  offeror's  track 
record  and  methodology  are  checked  through 
contact  with  administrative  offices  for 
the  prime  and  also  for  the  major  sub¬ 
contractors.  The  team  approach  is  vital 

With  respect  to  Public  Law  95-507,  con¬ 
cerning  small  and  disadvantaged  businesses 
utilized  by  a  prime,  the  pver  zealous 
support  of  the  law  should  not  jeopardize 
the  mission  assurance  requirements  of  the 
program.  The  management  control  of  subs 
is  an  important  item  for  review  during 
evaluations.  Also,  where  mission  assurance 
is  cranked  into  an  award  fee  plan,  the 
evaluation  team  having  that  specific 
expertise,  carefully  assess  all  the  impli¬ 
cations  of  that  inclusion.  By  the  same 
token,  any  alternate  proposal  for  satisfy¬ 
ing  mission  assurance  solicitation  re¬ 
quirements  is  given  the  same  quality  of 
evaluation,  utilizing  the  same  criteria, 
as  for  the  basic  proposal.  This  evalua¬ 
tion  is  presented  to  the  decision  maker 
with  all  other  trade-offs  which  may  be 
advantageous  to  the  Government. 

I  have  deliberately  included  today  the 
basic  evaluation  technique  to  assure  you 
that  mission  assurance  elements  sometimes 
called  "illities"  and  software  receive 
the  same  thorough  analysis  as  all  other 
elements  of  a  proposal.  At  Space  Division 
we  are  steeped  with  the  adage  that  for  the 
want  of  a  nail  the  shoe  was  lost, for  the 
want  of  a  shoe  the  horse  was  lost  and  for 
the  want  of  a  horse  the  battle  was  lost. 
There  isn't  a  system  deployed,  launched 
or  operated  by  this  division,  or  an  opera¬ 
tional  command,  which  doesn't  require 
perfection  of  the  highest  order.  Ground 
control  and  communication  systems  must  be 
as  reliable  as  their  space  borne  cousins. 
The  ground  items  can  be  repaired  but  back¬ 
up  and  alternates  must  be  provided.  Their 
procedures  and  design  must  keep  a  firm  eye 
on  the  nail  for  that  important  shoe. 

Since  the  evaluation  techniques  are  widely 
known  and  procedures  or  ground  rules  are 
public  knowledge,  each  offeror  in  a  com¬ 
petition  can  evaluate  himself  as  well  as 
the  surmised  approach  or  presentation  of 
his  competitors  and  assess  the  likelihood 
of  winning  a  competition.  Remember,  the 


low  bidder  doesn't  always  win.  And 
whether  or  not  a  solicitation  groups  all 
mission  assurance  elements  into  one 
volume  of  a  proposal  is  not  an  indication 
it  will  receive  more  or  less  attention 
from  the  decision  maker.  It's  the 
quality  of  the  response  that  gets  atten¬ 
tion,  not  the  physical  location  of  the 
presentation.  Is  your  approach  or  pre¬ 
sentation  really  better  than  that  which 
your  competitor  can  provide?  Only  each 
offeror  can  answer  that  question.  Yours 
is  not  an  easy  task,  namely  expanding 
10  or  20  pages  of  proposal  instructions 
into  4  or  800  pages  of  proposal,  and 
ours  is  not  an  easy  task,  condensing 
those  pages  into  a  one  to  four  hour 
briefing  and  written  analysis  of  the 
salient  facts.  Perhaps  in' the  discussions 
today  and  tomorrow  some  improvement  in 
the  evaluation  technique  and  presentation 
may  be  developed  which  will  benefit  both 
industry  and  the  Government. 
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PROPOSAL  EVALUATION 

Mr.  Clen  Neehan 
HQ  NASA 

INTRODUCTION 

I'd  like  to  start  out  by  saying  that  1 
find  the  concept  of  mission  assurance 
particularly  Important  to  NASA  as  an 
agency.  This  Importance  traces  back  to 
the  beginnings  of  NASA  when  the  mission 
of  the  agency  was  the  national  priority 
and  required  the  marshalling  of  much  of 
the  scientific,  engineering,  and  manage¬ 
ment  talent  In  the  country  Into  an  effort 
that  reached  Its  culmination  with  man's 
first  steps  on  the  moon.  This  effort 
required  an  approach  that  utilized 
the  expertise  of  government.  Industry, 
and  university  personnel  Is  developing  a 
workable  program  that  would  succeed  In 
accomplishing  the  assigned  mission.  Ihe 
program  also  fostered  within  NASA  an 
evaluation  approach  that,  notwithstanding 
the  shifting  of  national  priorities,  has 
carried  forth  to  the  present. 

Itie  evaluation  approach  has  always  had  as 
Its  main  focus  the  necessity  to  look  Into 
the  future  In  order  to  determine  the 
steps  that  must  be  taken  now  to  ensure  a 
successful  space  flight.  When  space  was 
an  unexplored  horizon,  each  step  was  a 
new  one  and  had  to  be  carefully  weighed 
before  proceeding  Into  the  next  step, 

Ihe  uncertainties  Inherent  In  this  ex¬ 
ploratory  process  coupled  with  the  factors 
of  prestige  gained  from  "firsts"  Is  space 
and  safety  for  manned  space  flights  tended 
to  override  the  Importance  of  cost  factors. 

The  evaluation  approach  presently  being 
used  still  maintains  the  main  tccus  of 
looking  Into  the  future,  however,  these 
has  been  subtle  shift  In  emphasis  to 
what  I  would  call  "cost  effective  space 
flight."  This  shift  has  had  an  effect  on 
the  evaluation  approach  but  has  not  forced 
a  change  In  It,  thereby  validating  the 
thought  processes  of  those  who  developed 
an  approach  that  has  remained  flexible 
and  responsive  to  the  changing  needs  of 
the  agency.  I  will  now  enter  Into  a  "nuts 
and  bolts"  discussion  of  this  approach  and 
comment  on  Its  development  and  Implementa¬ 
tion  within  NASA. 


SOURCE  EVALUATION  BOARDS  (SEBs) 

In  my  experience  as  a  contract  negotiator 
and  contracting  officer,  I  have  often 
characterized  the  source  evaluation  pro¬ 
cess  as  a  method  by  which  a  subjective 
Judgment  Is  made  as  objectively  as  possi¬ 
ble.  This  highly  simplified  definition 
helps  me  In  understanding  the  reasons  be¬ 
hind  the  various  source  selection  process¬ 
es  that  have  been  developed  at  different 
government  agencies  and  In  different  In¬ 
dustrial  concerns.  Although  many  similari¬ 
ties  exist  In  structure,  the  differences 
come  about  because  the  process  Is  basical¬ 
ly  one  of  declslon-maklng,  and  most 
organizations  differ  In  their  process  of 
making  decisions.  Ihus,  evaluation  boards 
normally  have  rigid  structural  require¬ 
ments  concerning  who  will  make  the 
decision,  what  regulations  or  laws  must  be 
followed,  what  areas  of  proposals  will  be 
scored,  etc.  Ihese  black  and  white  or  do 
and  don't  areas  are  fairly  similar  within 
organlz itlons  and  form  a  perimeter  within 
which  the  Source  Evaluation  Board  may 
operate.  Within  ••hat  perimeter,  however, 
the  Judgment  f  jrs  thot  lead  to  gray  or 
shaded  areas  of  distinction  take  prece¬ 
dence.  Iho  handling  of  these  gray  or 
shaded  areas  Is  the  factor  that  determines 
the  success  of  the  process  and  characteriz¬ 
es  It  to  those  outside  of  the  organization. 

Within  NASA,  the  formal  process  of  source 
evaluation  has  been  documented  In  the  NASA 
Source  Evaluation  Manual,  Ihc  following 
excerpts  are  taken  from  the  Preface  and 
Foreward  to  this  manual: 

"The  source  evaluation  and 
selection  process  covered 
by  this  manual  exemplifies 
our  efforts  to  emphasize 
the  application  of  sound 
judgment  to  the  problem  of 
source  evaluation." 

"This  manual  provides  general 
and  specific  guidance  ...  in 
the  evaluation  of  competing 
companies  and  their  proposals 
In  negotiated  procurements." 
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"Its  (the  manual)  Intention  Is  to 
encourage  the  exercise  of  judgment 
In  the  many  Important  aspects  where 
It  Is  essential,  and  to  prescribe  a 
set  process  where  experience  has 
shown  It  to  be  required." 

( 

"Users  (of  the  manual)  are  expected 
to  apply  common  sense  In  determining 
where  appropriate  variations  and 
adaptations  are  necessary  In  Indlvl- 
ual  situations,  provided  that  these 
do  not  constitute  a  departure  from 
basic  concepts  and  Intent." 

"The  Manual  Is  organized  so  as  to 
provide  substantive  and  procedural 
guidance. " 

As  can  be  seen  by  these  excerpts,  there 
Is  n  realization  of  the  gray  areas  In 
which  an  SEB  operates,  a  realization  that 
allows  for  a  different  character  to  be 
demonstrated  by  each  Individual  SEB. 

Within  an  evaluation  process,  there  are 
several  key  Indicators  as  to  the  direction 
that  the  Board  will  take  or  the  areas  that 
the  Board  will  consider  Important.  Of 
course,  the  evaluation  factors  and  their 
relative  order  of  Importance  are  keys  to 
Individual  procurements  but,  on  a  genera¬ 
lized  level,  the  composition  of  Boards 
and  their  place  In  the  declslon-maklng 
process  emphasizes  the  policies  of  the 
organization.  At  NASA,  line  management 
Is  specifically  tasked  with  responsibili¬ 
ties  for  declslon-maklng  while  line  and 
staff  management  work  together  to  ensure 
proper  functioning  of  the  process.  Thus, 
one  could  expect  an  evaluation  bias 
weighing  heavily  on  the  technical  aspects 
of  a  proposal.  This  bias  is  further  re¬ 
inforced  by  the  following  excerpt  from 
duties  of  the  Board: 

"Evaluate  the  varlour  features 
of  the  proposals  that,  together, 
determine  how  well  each  proposer 
might  fulfill  mission  suitability 
requirements;  and  to  combine  these 
Judgments  Into  an  Integrated 
assessement  of  relative  probable 
performance." 


The  Boards  are  comprised  of  an  appropriate 
mix  of  qualified  management,  technical, 
scientific,  contracting,  and  business 
experts  and  shall  normally  not  exceed 
seven  voting  members.  Including  the 
Chairman.  However,  it  is  desirable  that 
voting  members  of  the  Board  Include  people 
who  will  have  key  assignments  on  the  pro¬ 
ject  to  which  the  procurement  Is  directed 
Finally,  there  arc  normally  two  major 
committees  set  up  within  an  SEB,  a  techni¬ 
cal  committee  to  evaluate  and  score 
proposals  against  mission  suitability 
criteria  and  a  business  management 
committee  to  study  and  analyze  costs, 
evaluate  experience  and  past  performance, 
and  analyze  any  other  factors  that  may  be 
pertinent  to  the  selection.  The  Board 
does  not  make  recomncndatlons  to  the 
Source  Selection  Official,  It  reports 
findings  that  are  scored  for  mission 
suitability  and  commented  on  for  cost  and 
other  factors.  Although  the  fact  that 
mission  suitability  Is  scored  should  not 
be  overrated,  since  the  Selection  Official 
must  consider  all  factors  In  making  the 
selection.  It  remains  a  prime  factor  In 
determining  the  likelihood  that  an  offeror 
can  deliver  a  product  that  will  meet  the 
Government's  stated  requirements. 

SEB  Operating  Procedures 

The  official  activities  of  a  Board  for  a 
particular  procurement  will  comnence  upon 
receipt  by  the  Chairman  of  the  letter 
establishing  the  Board  and  designating  Its 
members.  The  Board  then  carries  out 
administrative  tasks  to  ensure  that 
members  understand  the  procurement,  that 
adequate  personnel  and  expertise  are  avail¬ 
able,  that  all  prior  relevant  work  Is 
collected,  and  that  a  list  of  qualified 
sources  Is  developed.  The  Board  then 
reviews  In  detail  the  paperwork  generated 
by  the  project  office  that  Includes  the 
procurement  plan,  project  plan,  and  all 
areas  of  the  RFP.  The  primary  objective 
at  this  time  Is  to  ensure  consistency  of 
of  requirements  In  all  steps  leading  up  to 
the  release  of  the  RFP,  After  release  of 
the  RFP,  a  pre-proposal  conference  may  be 
held.  Upon  receipt  of  proposals,  the 
contracting  officer  sends  all  proposals. 
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unopened,  to  the  SEB  Chairman.  The  Board 
convenes  promptly  and  the  evaluation 
begins,  normally  Involving  the  committees. 
The  committees  will  first  identify  any 
unacceptable  proposals  which  will  receive 
no  further  evaluation  and  then  make  a 
detailed  review  of  the  acceptable  pro¬ 
posals,  the  results  of  which  will  be 
presented  to  the  Board.  The  Board  will 
then  make  a  competitive  range  determina¬ 
tion  and  conduct  a  finhl  evaluation  to 
arrive  at  findings  to  be  presented  to  the 
Source  Selection  Official  with  final 
scoring  on  mission  suitability  factors 
and  reports  on  the  analyses  of  cost  and 
other  factors. 

Considerations 

Prior  to  the  appointment  of  an  SEB  Chair¬ 
man  and  Board  Members,  there  has  been  an 
undeflneable  period  during  which  basic 
and  advanced  research  has  been  conducted 
and  has  pointed  out  a  need  that  can  be 
satisfied  through  the  use  of  space  tech¬ 
nology  or  that  state-of-the-art  advances 
in  space  technology  can  satisfy.  At  NASA, 
these  normally  take  the  form  of  new  or 
improved  instruments,  use  of  spacecraft  to 
collect  data,  or  transmission  of  data  from 
space  to  users  of  that  data.  One  of  the 
first  considerations  facing  an  SEB  is  the 
assimilation  of  that  prior  research.  This 
is  not  a  formidable  task  since  selection 
of  members  for  the  SEB  considered 
familiarity  with  the  project.  However, 
the  Board  must  evaluate  its  own  ability  to 
handle  all  facets  of  the  upcoming  procure¬ 
ment  and  ensure  that  It  has  sufficient 
expertise  to  cover  these  facets. 

The  requirements  review  process  takes  place 
place  in  two  stages.  The  first  of  these 
is  a  consideration  of  program  goals  as 
developed  from  a  technical  and  fiscal 
standpoint.  The  second  is  a  detailed 


review  of  the  SOW  and  RFP  to  ensure  that 
they  conform  to  these  goals.  Within  the 
RFP,  the  evaluation  criteria  and  their 
relative  order  of  importance  are  prime 
Indicators  of  where  the  Board  has  placed 
their  emphasis.  NASA  normally  uses  a 
1000  point  scoring  system  with  the  major 
mission  suitability  areas  being:  (1) 
understanding  the  requirement;  (2)  manage 
ment  plan;  (3)  excellence  of  proposed 
design;  (4)  key  personnel;  and,  (5) 
corporate  or  company  resources.  Each  of 
these  areas  is  weighted  by  the  Board  as 
they  develop  a  general  plan  for  scoring 
that  will  be  used  by  the  committees. 

There  is  no  format  for  scoring  plans  and 
they  can  vary  for  each  SEB.  Weightings 
are  not  disclosed  below  the  level  of  the 
SEB.  The  Board  also  reviews  the  unscored 
areas  of  cost,  experience  and  past  per¬ 
formance,  and  other  factors  to  determine 
if  there  is  any  specific  information  that 
may  be  required  in  the  proposals  over  and 
above  that  normally  required. 

The  considerations  of  the  period  after 
receipt  of  proposals  are  the  initial  and 
final  evaluations.  The  final  evaluation 
will  normally  include  written  or  oral 
discussion,  identification  of  the 
strengths  and  weaknesses  of  each  proposal 
identification  of  the  capabilities  of  the 
offerors,  and  the  best  collective  and 
objective  judgment  of  the  SEB  in  arriving 
at  its  findings. 


SUMMARY  OF  ISSUES 
AND  RECOMMENDATIONS 


RFP/SOW  COMMUNICATIONS 
SUMMARY  OF 

ISSUES  AND  RECOMMENDATIONS 


1.  Early  Interchange 
Issue 

Information  exchange  on  major 
contracts  should  be  initialed 
earlier. 

Recommendations 

oASFC  to  establish  formal  guide¬ 
lines  for  all  buying  divisions  to 
implement  industry  conferences 
to  promote  interchange  of  infor¬ 
mation  with  industry  prior  to 
development  of  business  strategy 
for  PMD  (Program  Management 
Directive). 

o  Early  release  of  draft-RFPs  and 
industry  participation  in  "Murder 
Boards"  should  be  continued. 

2.  Proposal  Evaluation 
Issue 

Proposal  evaluation  criteria  is  not 
sufficiently  stated  in  RFPs  for  the 
contractor  to  understand  program 
priorities. 

Recommendations 

o  Procurement  evaluation  panels 
should  insure  evaluation  criteria 
contains  sufficient  detail  to  enable 
the  contractor  to  properly  pre¬ 
pare  his  proposal.  Continued 
participation  of  industry  in  RFP 
murder  boards  will  assist  in 
achieving  this  objective. 

3.  Bidders  List 
Issue 

Utility  of  bidders  lists  and  their 
cost  effectiveness  is  questioned. 
There  is  too  much  duplication  of 


resources  involved  in  the  individ¬ 
ual  maintenance  of  local  bidder 
lists. 

Recommendation 

o  Recommend  the  local  bidders 
lists  be  replaced  by  SBA's  pro¬ 
curement  automated  source 
system  (PASS)  for  small  busi¬ 
nesses  and/or  other  centrally 
maintained  lists. 

o  Eliminate  bidders  lists  for  large 
businesses  or  centralize. 

4.  Use  of  CBD's  (Commerce  Business 
Daily) . 

Issue 

CBD  should  be  used  more  exten¬ 
sively  to  facilitate  early  govern¬ 
ment/industry  communications. 
Recommendations 

o  Buying  office  should  be  instructed 
to  use  the  CBD  as  the  primary 
device  for  soliciting  responses 
to  RFP's.  Contents  of  CBD 
submissions  should  be  sufficiently 
specific  to  adequately  inform  in¬ 
dustry  of  the  nature  of  the  pro¬ 
curement. 

o  Consideration  should  be  given  to 
electronically  transmit  CBD 
submissions  via  the  GSA  admini¬ 
strative  reporting  system. 

5.  Zero  Base  RFP/SOW 
Issue 

The  present  practice  of  specifying 
specifications  and  standards  are 
apparently  inhibiting  contractor 
responses  with  respect  to  how 
they  are  applied.  They  evidently 
feel  that  they  will  be  considered 
non- responsive  to  the  RFP,  conse¬ 
quently  the  government  is  not  re¬ 
ceiving  contractor  inputs  with  re¬ 
spect  to  tailoring  specification 
requirements. 

Recommendations 

o  Proposal  instructions  for  draft 


RFP's  should  adopt  a  zero  base 
concept  of  applying  contract 
specification  and  standards  which 
essentially  asks  the  contractor  to 
provide  his  tailored  application  of 
of  each  applicable  specification 
and  standard  that  1^  feels  are 
necessary  to  meet  the  essential 
needs  of  the  program/contract. 

6.  Lower  Tier  Specifications 
Issue 

The  preponderence  of  lower  tier 
specification  that  are  applied  to 
government  contracts  through  the 
'referencing  down”  procedure  is 
causing  much  'gold  plating”  and 
creating  now  essential  work  by 
contractors.  The  SPOs  do  not 
have  the  time  or  manpower  to 
review  the  lower  tiers  specification 
to  the  level  required  for  adequate 
application. 

Recommendations 

oA  project  should  be  initiated  at 
DOD  level  to  develop  a  specifi¬ 
cation/standards  traceability 
matrix  (preferably  an  automated 
program)  which  would  identify  the 
relationship(s)  of  all  lower  tier 
specification  to  the  extent  they 
are  applicable  to  the  parent  spec¬ 
ification  and  the  identification  of 
redundent  requirements. 

7.  Correlation  of  Contract  Require¬ 
ments. 

Issue 

RFPs  are  not  properly  organized 
for  ease  of  proposal  preparation 
and  evaluation. 


8.  Program  Background 
Info  rmation 

Issue 

The  government  rationale  for 
requirements  are  frequently  not 
made  known  to  contractors. 

Recommendations 

o  Buying  offices  should  be  encour¬ 
aged  to  establish  a  program 
library  of  information  studies, 
and  analyses  which  form  the 
basis  for  requirements.  Con¬ 
tractor  accessability  to  be  avail¬ 
able  when  CBD  announcement 
released. 

9.  Lessons  Learned 
Issue 

Government  inter-agency  commu¬ 
nications  are  too  limited.  Inter- 
organizational  lessons  learned  are 
not  readily  available  and  therefore 
not  used.  Contract  administra¬ 
tions  offices  can  be  a  part  of  the 
acquisition  team  to  facilitate  early 
communication  with  and  about 
contractors. 

Recommendations 

o  Contract  administration  services 
(CAS)  and  buying  divisions  should 
emphasize  the  desireability  of 
contract  administration  offices 
(CAO's)  receiving  draft  RFPs. 

o  Feasibility  of  establishing  a 
centralized  lessons  learned  data 
bank  be  investigated. 


Recommendations 

oSOW,  WBS,  evaluation  criteria, 
and  volume  organization  should 
all  be  correlated.  Proposals 
should  be  formatted  with  same 
correlation.  Requirements  should 
should  be  listed  by  priority, 
cirticality  and  tradeability. 
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Princeton 

Introduction  and  Need  for  Training 

Good  quality  Is  good  business  and  good 
politics.  Ignoring  quality  Invites 
higher  reject  rates  and  attendant  higher 
costs  thereby  risking  loss  of  business. 
After  all  the  recognized  disciplines  of 
reliability  have  been  observed  during  the 
design  phase,  one  must  still  insure  that 
the  product  is  built  as  intended  by  the 
design.  There  are  many  ways  to  do  this, 
but  this  paper  concerns  the  Importance  of 
training  in  promoting  quality  and  the  role 
that  training  plays  In  both  motivating  the 
employee  and  promoting  an  atmosphere  of 
quality  awareness.  The  Training  and  In¬ 
doctrination  Program  used  at  RCA  Astro  Is 
used  as  the  vehicle  to  illustrate  these 
concepts. 

With  the  complexity  of  the  designs  in  the 
space  Industry,  resulting  from  Increased 
functional  demands  coupled  with  more 
stringent  space,  weight,  and  power  re¬ 
quirements,  greater  demands  have  been 
placed  on  skills  of  manufacturing  and  test 
personnel.  The  Increased  demands  have 
placed  a  premium  on  building  the  quality 
Into  the  product,  checking  the  quality  of 
the  product,  and  measuring  the  quality 
trends.  After  the  design  process  Is  com¬ 
pleted,  the  building  of  a  quality  product 
requires  sound  procedures,  good  equipment, 
a  cooperative  management  attitude,  a  con¬ 
ducive  work  environment,  good  material, 
and  skilled  people.  The  people  are  the 
integrating  element  of  all  these  ingre¬ 
dients.  It  follows,  then,  that  to  Improve 
quality,  the  people  must  be  skilled,  aware 
of  quality,  and  properly  motivated. 

A  well  planned  Training,  Certification, 
and  Indoctrination  Program  improves 
employee  skills.  Increases  the  employee's 
awareness  of  quality,  may  enhance  employee 
motivation,  and  Improves  productivity  and 
quality  for  the  company.  A  good  training 
program  must  satisfy  the  Individual's  fun¬ 
damental  psychological  needs  and  human  mo¬ 
tivation  desires.  When  these  requirements 
are  met,  greater  Job  satisfaction  and 
reduced  personnel  turnover  occurs. 
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Training  1s  Invoked  most  frequently  on 
the  new  employees  or  for  an  employee 
Intending  to  perform  specific  processes 
for  the  first  time.  The  purpose  of  this 
training  Is  to; 

1.  Provide  the  Individual  with  the  skills 
and  knowledge  necessary  to  certify  for 
processes  that  are  critical  to  product 
quality  such  as  soldering,  welding, 
and  crimping  operations. 

2.  Impart  specific  skills  required  to 
fabricate,  inspect  and  test  a  quality 
product. 

3.  Provide  appreciation  for  the  Impor¬ 
tance  of  Individual  operator  tasks. 

4.  Emphasize  the  Importance  of  group  or 
team  effort  for  quality  and  produc¬ 
tivity. 

Indoctrination  as  used  here  Is  Invoked 
most  frequently  on  current  employees.  The 
purpose  of  this  Indoctrination  1s  to: 

1.  Provide  the  employee  an  understanding 
of  division  policy  and  approaches  to 
achieve  and  accomplish  specific  tasks 
and  objectives. 

2.  Make  employees  aware  of  changes  to 
specific  policies,  objectives,  or 
procedures. 

3.  Promote  awareness  for  quality  and/or 
other  Items  of  current  concern. 

Both  the  Training  and  Certification  of 
operators  and  testers,  and  the  Indoctri¬ 
nation  of  employees  In  specific  topics  are 
the  responsibility  of  Quality  Engineering. 

Training  and  Certification  Program  for 
Operators  and  Testers 

The  successful  completion  of  job  related 
Training  and  Certification  courses  Is 
mandatory  at  Astro  for  manufacturing 
operators  and  test  personnel  of  flight 
hardware. 
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Formal  training  courses  are  provided  by 
Astro's  Training  Center  to  teach  employees 
the  skills  necessary  to  certify  for  spe¬ 
cific  operations. 

The  Training  and  Certification  program 
used  at  RCA  Astro  has  the  following 
features  and  requirements: 

1.  Formally  required  by  Division  Policy 
In  the  Division  Operating  Instructions. 

?.  Training  In  selected  areas  for  certif¬ 
ication. 

3.  Successful  completion  is  a  condition 
for  employment. 

4.  Training  consists  of: 

a.  An  overview  stressing  Importance 
of  specific  skills  to  the  end  pro¬ 
duct  and  to  the  division. 

b.  Detailed  Instruction  (demonstra¬ 
tion  of  how  to)  by  Instructor. 

c.  Practical  experience  - 
Individual's  performance  of  task. 

d.  Critique  of  Individual's  perfor¬ 
mance  stressing  weak  points  or 
commending  good  performance. 

5.  Initial  certification  Is  limited  to  a 
trial  period  of  60  days  to  assess 
work  In  the  assembly  environment. 

6.  Permanent  certification  by  certifica¬ 
tion  card. 

Certification  Is  recognized  only  after 
the  operator  has  achieved  satisfactory 
Job  performance  for  sixty  days  as  deter¬ 
mined  by  his  supervisor.  Training  Center 
personnel,  and  by  the  Defect  Summary 
which  Is  discussed  later. 

A  variety  of  courses  are  offered  by  the 
Training  Center  at  RCA  Astro  to  achieve 
the  desired  results.  Each  manufacturing 
operator  must  take  a  basic  40  hour  Train¬ 
ing  Course  called  Descriptive  Assembly 
Fundamentals  plus  one  other  course  for 
the  specific  area  where  the  operator  Is 
assigned.  The  basic  course  Is  designed 
to  teach  the  operator  the  rudimentary 
knowledge  and  skills  he  needs  to  be  em- 
r.loyed  In  RCA  Astro's  Product  Operations. 


The  course  also  attempts  to  motivate  the 
operator  by  Instilling  pride  In  his  work, 
and  by  showing  him  how  his  efforts  con¬ 
tribute  and  are  Important  to  the  final 
product. 

The  courses  are  taught  on  site  In  a  desig¬ 
nated  training  area  secluded  from  the 
normal  hustle  encountered  In  the  manufac¬ 
turing  environment.  Each  operator  Is 
Issued  a  set  of  tools,  materials,  and 
shown  how  to  use  them  at  a  work  bench  sim¬ 
ilar  to  those  used  In  the  manufacturing 
assembly  area.  The  learning  experience 
Is  primarily  a  work  experience  where  the 
operator  Is  taught  to  utilize  specific 
processes  or  operations.  The  classes  are 
limited  to  ten  student  operators  so  that 
the  Instructor  can  provide  Individual 
attention  as  required. 

A  list  of  the  most  frequently  offered 
courses  Is  shown  In  Table  1.  Several  of 
the  courses  shown  such  as  Operation  of 
Overhead  Cranes  and  Multipin  Connector 
Mating/Demating  are  for  Testers  rather 
than  manufacturing  assembly  operators. 

Throughout  the  Instruction  period  the 
employee's  work  Is  critiqued  and  he  1s 
shown  how  to  Improve  his  quality.  The 
employee  Is  encouraged  to  ask  questions 
and  repeat  any  steps  he  feels  need  Im¬ 
provement.  The  training  sessions  are 
arranged  to  Include  some  training  films, 
slides,  or  lecture  type  Instruction  away 
from  the  work  benches. 

For  each  student  the  Instructor  completes 
a  Training  Report  as  shown  In  Figure  1. 
Each  Item  on  this  report  Is  discussed 
with  the  student.  A  "Satisfactory"  must 
be  obtained  for  each  Item  for  the  student 
to  be  certified.  Following  the  completion 
of  the  course,  the  student  either  receives 
a  Temporary  Certification  valid  for  60 
days  or  Is  Judged  as  not  qualified  for 
certification.  In  the  latter  case,  the 
student  cannot  be  considered  for  employ¬ 
ment  as  a  manufacturing  assembly  operator. 
With  a  "Satisfactory"  completion  the  stu¬ 
dent  Is  given  a  Certification  Card  as 
shown  In  Figure  ?.  This  Initial  card  will 
have  an  expiration  date  of  60  days. 

Following  completion  of  the  training 
courses,  the  student  Is  released  to 
Product  Operations  for  on  the  Job 
experience  and  training. 
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TABLE  1.  TYPICAL  CERTIFICATION  COURSES  OFFERED  BY  TRAINING  CENTER 


Course  Recertification 

Hours 

Hours 

Course  Title 

40 

Descriptive  Assembly  Fundamentals 

40 

16 

Hand  Soldering  Fundamentals  NHB6300.4  (3A) 

16 

8 

Soldering/Wiring  Verification 

8 

1 

Close  Circuit  Reflow  Soldering 

4 

- 

Operating  of  Overhead  Cranes 

40 

4 

Microcircuit  Soldering 

16 

1 

Wave  Soldering 

8 

1 

Solar  Cell  Assembly  Interconnections 

4 

1 

Solder  less  W1 rewrap 

32 

1 

Modular  Induction  Brazing 

40 

2 

Semi-Automatic  Buttonhead  Welding  Fundamentals 

40 

2 

Manual  Buttonhead  Welding  Fundamentals 

40 

2 

Resistance  Cross  Wire  Welding 

16 

2 

Parallel  Gap  Welding 

2A 

2 

Back  Plane  Welding 

40 

4 

Plastics  Application  Technology 

As  Req'd 

As  Req'd 

On-the-Job  Training  Format 

?4 

4 

Buttonhead  Welding  (Inspector) 

4 

- 

Multi  pin  Connector  Mating/Demating 

The  new  employee  is  encouraged  to  keep  1n 
contact  with  the  Training  Center  and  ask 
questions,  If  he  is  having  difficulty  or 
is  having  a  confidence  crisis.  Training 
personnel  also  make  It  a  point  to  visit 
the  now  employees  on  the  Product  Opera¬ 
tions  floor  to  see  how  they  are  doing. 
After  sixty  days  of  on-the-Job  training. 

If  the  results  are  still  satisfactory,  the 
new  employee  Is  given  a  permanent  Certifi¬ 
cation  Card. 


This  certification  remains  In  effect  until 
an  event  occurs  that  requires  the  employee 
to  be  recertified.  Recertification  Is 
mandatory  If  any  one  of  the  following 
events  occur: 

1.  Quality  of  work  unsatisfactory  In 
specific  Instances. 

2.  Operator  defect  sunmary  Indicates  an 
unsatisfactory  trend. 

3.  Work  Interruption  for  greater  than  30 
days. 

4.  New  techniques  developed  which  require 
different  skills. 

5.  Operator  upgraded  to  an  Inspector. 


Maintaining  Product  Quality  and  Quality 
Awareness 

After  the  employees  are  fully  trained, 
both  the  progress  of  each  operator  and 
the  product  quality  Is  constantly  as¬ 
sessed.  This  Is  done  by  the  use  of  the 
following  techniques: 

1.  Product  Is  built  to  certified  processes. 

2.  Product  is  Inspected  at  key  toll  gates 
by  certified  Inspectors. 

3.  Follow-up  Inspections  are  performed  by 
Training  Center  personnel. 

4.  Flow  of  product  to  process  steps  and 
Inspection  points  Is  defined  In  a  Fab¬ 
rication  Procedure  Record  (FPR). 

This  Is  a  controlled  specification. 

5.  Deviations  are  documented  per  a  Flow 
Alteration  Record  (FAR)  prior  to  Im¬ 
plementation. 

6.  Testing  of  product  per  a  test  proce¬ 
dure,  which  Is  also  a  controlled  spec¬ 
ification. 

7.  Defect  reporting  system  for  Inspection 
results. 

8.  Test  discrepancy  reporting  system  for 
testing  results. 


The  above  techniques  are  designed  to  be 
formal  and  to  keep  each  employee  aware  of 
the  Importance  for  good  quality.  In 
addition  to  the  above  Indoctrination 
sessions  for  specific  topics  of  concern 
are  given  to  selected  groups  when  war¬ 
ranted.  Specific  examples  Include: 

1.  The  showing  of  quality  oriented 
training  films  when  recommended  by 
the  Training  Center  or  Management. 

2.  Specific  topics  of  concern  to  the  Pro¬ 
gram  Management  Offices. 

At  each  Inspection  point.  Quality  Control 
Inspects  the  product  or  process  step  and 
determines  Its  acceptability  to  specified 
requirements.  If  the  Inspected  Item  Is 
judged  not  acceptable,  then  a  Unit  Eval¬ 
uation  Report  (UER)  Is  prepared.  If  the 
defect  Is  judged  to  be  valid  by  the  QC 
leader,  the  defect  Is  discussed  with  the 
Manufacturing  Leader  and  the  Operator. 

At  the  discretion  of  the  QC  leader  the  de¬ 
fect  may  also  be  reviewed  by  Quality  Engi¬ 
neering.  The  defect  Is  also  Inputted  Into 
a  Computer  Data  Bank  for  later  recall  and 
Management  visibility.  The  above  follows 
the  flow  shown  in  Figure  3. 

a.  Quality  Awareness  via  Defect  Reporting 

The  Defect  Reporting  System  shown  In 
Figure  3  has  been  found  to  be  very  effec¬ 
tive  as  used  at  RCA  Astro.  It  has  the 
following  salient  features: 

1.  Operator  performance  is  available  on 
any  given  task. 

2.  Training  personnel  can  approach 
operator  Independently. 

3.  Promotes  team  approach  rather  than  an 
Individual  contest. 

4.  Provides  management  with  various 
analyses: 

e  By  operator 

•  By  Inspector 

f  By  work  center 

•  By  program 

•  By  Item  manufactured 

e  Accept/reject  per  total 
Inspections 

5.  Provides  data  for  development  of 
future  plans. 


Computer  reports  are  Issued  on  both  a 
weekly  and  a  monthly  basis.  The  weekly 
reports  are  very  detailed  and  specific  re¬ 
garding  the  defects  Incurred.  The  monthly 
reports  are  formatted  to  provide  visibil¬ 
ity  of  trends  Indicative  of  problems  that 
might  be  occurring.  These  reports  provide 
a  data  base  so  that  Quality  Enqineering 
or  Product  Dperatlons  may  Institute 
corrective  actions. 

b.  Quality  Awareness  via  Test 
Discrepancy  Reporting 

After  the  product  Is  built,  and  Inspected, 
It  Is  subjected  to  Its  Flight  Acceptance 
Testing  and  all  test  discrepancies  are 
formally  reported.  Each  discrepancy  Is 
formally  acted  upon  and  dispositloned  by 
a  Test  Review  Board.  The  discrepancy  Is 
then  coded  and  Inputted  Into  a  computer 
data  bank  so  that  various  trends  can  be 
detected.  The  TDR  System  Is  not  primarily 
a  barometer  for  measuring  training  effec¬ 
tiveness  and  Is  only  mentioned  here  for 
completeness.  Nevertheless,  many  useful 
trends  have  been  detected  by  this  system 
and  as  a  result  changes  In  processes  and 
new  training  courses  have  been  Imple¬ 
mented.  As  an  example  on  several  major 
programs  It  was  noted  that  an  excessive 
number  of  multipin  miniature  connector 
failures  were  being  encountered  in  the 
Integration  area.  The  failures  were 
broken  pins,  bent  pins  and  chipped  Insu¬ 
lation.  Analyses  were  conducted  and  It 
was  concluded  that  the  problem  was  not  a 
generic  parts  problem  but  operator  re¬ 
lated.  It  was  decided  to  have  the 
Training  Center  develop  a  Connector  Mate/ 
Oemate  course  in  which  selected  Test  Qper- 
ators  were  certified  and  the  mate/demate 
of  connectors  was  restricted  to  the 
certified  test  operators.  The  discrepan¬ 
cies  immediately  and  dramatically  dropped 
to  a  lower  plateau.  Even  though  the  Im¬ 
provement  was  gratifying  each  subsequent 
discrepancy  was  a  constant  source  of  Irri¬ 
tation  to  the  Program  Qffice.  So  addi¬ 
tional  steps  were  Instituted  which 
required: 

1.  A  Mate/Demate  Log 

2.  Dbservance  of  the  Mate/Demate  by  the 
Test  Leaders,  and 

3.  A  Quality  Control  check  of  all  con¬ 
nectors  mated  and  demated  before 
power  turn-on. 


The  result  of  instituting  a  Training 
Course  for  Certification  plus  surveillance 
dropped  the  discrepancies  to  essentially 
zero. 

Quality  Awareness  Indoctrination  Sessions 


depending  upon  audience  reaction.  The 
goal  is  to  make  each  topic  available  once 
a  year,  unless  events  warrant  more 
frequent  scheduling.  Attendence  at  a 
session  for  any  one  topic  is  generally 
limited  to  twenty-five  people. 


As  previously  mentioned  in  addition  to 
formal  Training  courses  Intended  for 
Certification  of  operators,  there  is  also 
a  regular  program  of  quality  awareness 
indoctrination  sessions.  These  sessions 
are  primarily  for  engineering  and  test 
personnel  and  are  intended  to: 

1.  Make  the  employee  aware  of  Division 
policy  on  selected  topics  important 
to  his  specific  assignments. 

2.  Provide  specific  information  as  to  why 
certain  practices  are  necessary,  e.g. 
clean  areas. 

3.  Instruct  employees  on  how  to  perform 
specific  tasks,  e.g.  complete  Test 
Discrepancy  Reports,  Engineering 
Change  Notices. 

4.  Provide  information  when  major 
changes  occur  in  Division  Operating 
Instructions. 

5.  Provide  feedback  when  a  Program  has 
immediate  needs  or  concerns. 

6.  Impart  an  attitude  of  Quality  Aware¬ 
ness  to  the  employee, 

7.  Obtain  feedback  from  the  employee. 

These  sessions  are  the  responsibility  of 
Quality  Engineering.  They  are  held  in  a 
relaxed  environment  where  employees  are 
encouraged  to  ask  questions,  express 
their  opinions,  and  explain  any  diffi¬ 
culties  encountered.  Each  session  gener¬ 
ally  lasts  about  one  to  two  hours  each 


The  Quality  Awareness  training  sessions 
have  been  found  to  be  an  important  source 
of  employee  suggestions  to  Improve  proce¬ 
dures  where  they  have  encountered  pro¬ 
blems.  The  success  of  the  program  has 
resulted  in  many  employee  suggestions, 
which  have  been  Incorporated  in  subsequent 
revisions  of  the  Division  Operating 
Instructions. 

Table  2  lists  some  of  the  typical  Quality 
Awareness  sessions  regularly  offered  by 
Quality  Engineering. 


Summary 

Quality  awareness  and  motivation  can  be 
provided  and  enhanced  by  training.  This 
requires  a  planned  approach  and  an  atmos¬ 
phere  conducive  to  employee  pride  of  work¬ 
manship.  The  starting  point  is  formal 
courses  for  training  in  basic  skills  and 
certification  in  critical  processes.  In 
addition  quality  awareness  sessions  are 
required  to  make  employees  aware  of  policy 
procedures.  These  are  required  on  a  reg¬ 
ular  basis  and  on  special  occasions  when 
warranted  for  special  topics. 

After  the  training  and  certification  a 
continuous  surveillance  is  required.  The 
checks  provided  by  the  implementing  of  a 
Defect  Reporting  System  and  the  Test  Dis¬ 
crepancy  Reporting  System  provides  a  data 
base  which  is  useful  to  both  management 
for  visibility  and  planning,  and  to  the 
employee  for  maintaining  his  motivation 
and  as  a  constant  reminder  to  be  aware  of 
quality. 


TABLE  2.  TYPICAL  INDOCTRINATION  SESSIONS  OFFERED  BY  QUALITY  ENGINEERING 

Topic 

RECEIPT,  INSPECTION  &  STORAGE  OF  FLIGHT  TYPE  GFE 
CONTROL  OF  CONFIGURATION  AND  DESIGN  TECHNICAL  DOCUMENTS 
ENGINEERING  CHANGE  &  STOP  WORK  NOTICES 
CONFIGURATION  MGMT  OF  PRODUCT  RELATED  COMPUTER  PROGRAMS 
FABRICATION  &  ASSEMBLY  CONTROL  DOCUMENTS 
INTEGRATED  SPACECRAFT  TESTING 

CONFIGURATION  ACCOUNTING  AT  SYSTEM  OR  SPACECRAFT  LEVEL 
REPORTING  OF  TEST  PROBLEMS  AND  DISCREPANCIES 
CONTROLLED  CLEAN  AREAS 

CERTIFICATION  OF  PERSONNEL  FOR  SPECIAL  PROCESSES 
PROJECT  OFFICE  INTERFACES  WITH  PRODUCT  ASSURANCE 
TEST  INTERCONNECTION  DEVICES 
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Training  Report 

RCA/Government  Systems  Division 

Astro  Electronics/Prirjceton,  New  Jersey 

ncii 

Course  Title 

Course  Number 

Startirrg  Date 

Length  in  Hours 

Other 

Last  First  Initial 

Employee  Number 

Activity  Number 

_ 1 

Location 

Supervisor 

Supervisor  Location 

Organization  (if  other  than  AE) 

Original 

Recertification 

Spedal  Certification 

School 

Written  Test  Grade 

Record  of  Demonstration 


Workmanship  Items 

Satisfactory 

Unsatisfactory 

(A) 

IB) 

(C) 

(Dl 

(El 

(FI 

IGI 

(HI 

III 

iji 

Comments  (To  Aid  in  Evaluating  Trainee's  Ability.  List  Trainee's  Work  Habits,  Weak  Points,  Etc.) 


Date  of  Completion 


Typed  Name  and  Signature  of  RCA  Instructor 


Certified  in 


I  1 60  Day  Temporary  Certification  Expiring  ||  |  Not  Qualified  for  Certification 
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QUALITY  CIRCLES  AT  HONEYWELL 
By  Dr.  Michael  Donovan 


It's  no  secret  that  the  United 
States  is  facing  a  real  producti¬ 
vity  crisis.  The  signs  are  every¬ 
where,  spiriling  inflation,  rising 
unemployment  and  lost  markets  to 
foreign  competition.  For  over  10 
years  our  nation  has'  been  on  the 
bottom  of  the  list  of  the  world's 
industrialized  nations  in  producti¬ 
vity  growth.  Organizations  in  all 
sectors  of  our  economy  are  feeling 
the  need  to  improve  productivity. 

Most  organizations  respond  to 
the  call  to  improve  productivity 
by  screw- tightening :  increasing 
management  control  over  things  and 
people.  But  at  Honeywell,  Inc., 
the  response  has  been  exactly  the 
opposite- -and  it's  paying  off. 
Instead  of  centralizing  responsi¬ 
bility  and  strengthening  controls 
to  solve  productivity  problems, 
Honeywell  has  been  slowly  de¬ 
centralizing  both  the  responsibil¬ 
ity  and  authority  for  productivity 
improvement . 

The  central  element  in  this  de¬ 
centralization  has  been  the 
introduction  of  a  Japanese  inven¬ 
tion  known  as  Quality  Circles. 
Quality  Circles  began  in  the  1960s 
as  part  of  Japan's  strategy  to 
establish  itself  as  a  leading  pro¬ 
ducer  of  high  quality  goods. 

The  Quality  Circle  concept  is 
simple,  but  difficult.  A  circle  is 
a  small  team  of  employees,  who  do 
similar  or  related  work,  who  meet 
regularly  to  identify,  analyze  and 
solve  work-related  problems.  Each 
circle  is  led  by  a  supervisor  or  a 
senior  employee  who  is  specially 
trained  in  problem  solving  disci¬ 
plines  and  group  leadership  skills. 
Simple.  The  difficult  part  is  the 
making  the  Quality  Circle  process  a 
way  of  life  in  an  organization- - 
where  employees  are  provided  with 
the  opportunity,  skills  and  leader¬ 
ship  to  participate  in  the  manage¬ 
ment  process . 


Honeywell  has  been  working  for 
over  six  years  to  create  an 
atmosphere  in  which  Quality 
Circles  can  become  a  way  of  life-- 
a  method  for  managing  people.  The 
company  was  one  of  the  first  U.S. 
companies  to  adopt  Quality  Circles, 
starting  its  first  circle  in  1974. 
Since  then,  the  program  has  grown 
to  over  160  circles  in  seven  major 
divisions  of  the  corporation, 
making  it  one  of  the  largest  and 
most  successful  programs  in  the 
country . 

Program  Goals 

Honeywell  managers  see  Quality 
Circles  as  a  key  element  in  their 
strategy  to  improve  both  producti¬ 
vity  and  the  quality  of  worklife. 
Their  objectives  are  to: 

1)  Improve  Productivity  Through: 

•  Early  identification  of 
problems , 

•  Quicker  and  more  knowled- 
gable  solutions  developed 
at  lower  levels, 

•  Innovations  in  the  methods 
of  conducting  the  business, 

•  Greater  distribution  of 
responsibility  for  achieving 
results , 

2)  Improve  Quality  of  Worklife 

Through: 

•  Greater  involvement  and 
participation  of  the  work¬ 
force  , 

•  Reinforcement  of  employee 
contributions  through  rapid 
response  to  ideas, 

•  Better  communication  and 
cooperation, 

•  Greater  identity  with  a 
work  team, 

•  Development  of  individual 
capabilities  to  solve  pro¬ 
blems  and  lead  groups. 

The  program  is  based  upon  the 
belief  that  employees  can  creat¬ 
ively  contribute  to  solving 
operational  problems  and  that  a 
truly  participative  role  is  rein¬ 
forcing  and  will  improve  employee 
satisfaction. 
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Circle  Identification 


Ingredients  for  Successful 
Implementation 

There  are  several  key  principles 
that  underlie  a  successful  pro¬ 
gram.  Figure  1.  illustrates  these 
principles . 
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Figure  1 . 

Management  Understanding 
and  Support 

For  the  program  to  be  success¬ 
ful,  management  must  see  Quality 
Circles  as  a  part  of  their  stra¬ 
tegy  to  achieve  operational  results. 
This  requires  a  thorough  orienta¬ 
tion  of  managers,  at  all  levels, 
to  the  Quality  Circle  philosophy 
and  practice.  In  addition,  manage¬ 
ment  needs  to  play  an  active  role 
in:  planning  the  program;  pro¬ 

viding  facilitation  and  coordination 
of  circle  activities;  listening 
and  insuring  honest,  rapid  responses 
to  circle  recommendations;  provid¬ 
ing  circles  with  information  and 
feedback;  and  focusing  the  circles 
on  important  organizational  goals. 

This  management  role  is  usually 
carried  out  through  a  steering  com¬ 
mittee  composed  of  key  managers 
from  the  unit  adopting  circles. 
Sometimes  this  committee  becomes  a 
"second- level  circle"  to  analyze 
and  solve  system-wide  problems 
that  go  beyond  the  authority  or 
capabilities  of  the  operating 
circles . 


The  process  begins  with  a  review 
of  the  existing  work  structure.  The 
objective  of  this  review  is  to 
identify  circles  where  people,  with 
common  goals  and  problems,  could 
benefit  from  disciplined  efforts  to 
solve  those  problems.  In  forming 
circles,  consideration  is  given  to 
factors  such  as simi larity  of  skills, 
physical  location,  flow  of  work, 
connected  functions  and  the  exist¬ 
ing  structures . 


Once  circles  are  identified. 


leaders  look  for  ways  to  highlight 
the  circle's  identity  and  build 
cohesion.  Some  circles  have  names 
such  as:  "The  Circuit  Board," 

"The  Pacers"  and  "Payable  Pencil 
Pushers."  Other  circles  have 
designed  tee-shirts,  jacket  emblems 
name  plate'  nnd  team  boards 
featurin  'r  photographs  of 

circle  me.uuis.  Participation  in  a 
highly  cohesive  circle  is  more 
likely  to  be  a  source  of  satis¬ 
faction  and  pride  to  its  members. 


Accountability  and  Feedback 
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The  regular  meetings  are  the 
means  for  aeliieving  the  goals  esta¬ 
blished  by  the  circle.  Meetings 
are  usually  held  every  other  week 
to  review  progress  on  action  items 
and  to  identify,  prioritize,  analyze 
and  solve  problems.  Quality  cir¬ 
cles  use  techni(iucs  like  I'orce- 
I'ield  Analysis;  Pareto  Analysis; 
Cause  and  liffcct  Analysis;  and 
Brainstorming  to  identify  and 
analyze  problems.  (Pigure  3.) 


The  focus,  however,  is  on  in¬ 
venting  workable  solutions  and 
taking  initiatives  to  implement 
those  solutions.  Tlic  "action 
register"  is  the  tool  the  circle 
uses  to  manage  the  implementation 
of  its  ideas.  It  is  merely  a  log 
that  documents  what  is  to  be  done, 
by  who,  and  by  when.  It  is  review 
cd  and  added  to  at  each  circle 
meeting.  Tlie  action  register  pro¬ 
vides  a  needed  discipline  and 
control  and  is  an  excellent  com¬ 
munications  tool  with  management. 

P o s i t i V c  Recognition 

As  each  circle  makes  progress, 
it  is  important  to  reinforce  their 
efforts.  Although  participation 
and  achievement  of  results  are 
inherently  rewarding,  circle 
leaders  are  encouraged  to  use  as 
many  forms  of  recognition  as 
possible.  Management  presentations 
newspaper  articles,  internal  con¬ 
ferences  and  awards  have  all  been 
successfully  used  to  reinforce 
circle  efforts. 

Training 

The  leader  is  crucial  to  the 
success  of  a  circle.  The  circle 
leader,  usually  a  first  line  super¬ 
visor,  is  responsible  for  planning 
the  meetings,  guiding  the  problem 
solving  process,  and  setting  the 
climate.  Supervisors  typically 
have  little  experience  working  with 
groups,  particularly  in  a  partici- 


Training  cont'd,.. 

pative  environment.  Our  training, 
therefore,  seeks  to  provide  not 
only  skills  but  experience. 

The  training  focuses  on  three 
skill  areas:  (1)  the  interpersonal 
skills  of  active  listening  and 
assertion;  (2)  planning  and  con¬ 
ducting  an  effective  circle  meeting, 
and  (3)  behavioral  rehearsal  of 
team  meetings  using  the  problem 
solving  techniques. 

The  greatest  time  is  spent  in 
behavioral  rehearsal,  using  video 
taped-critiqued  feedback  of  mock 
team  meetings.  In  small  classes  of 
8  to  10,  supervisors  are  prepared 
to  orient,  train  and  lead  their 
circles  in  problem  solving  activi¬ 
ties. 

The  training  is  conducted  in  a 
three-day  workshop  with  follow-up 
consultation  and  coaching,  on-the- 
job,  by  the  program  facilitator. 

In  addition,  circle  leaders  are 
provided  with  a  set  of  audio-visual 
modules  that  are  used  to  train  their 
teams  in  a  variety  of  problem 
solving  techniques  such  as  brain¬ 
storming  . 

The  Process 

The  first  circle  meeting  is 
spent  familiarizing  the  members 
with  the  Quality  Circle  concept  and 
answering  any  questions.  Next,  the 
circle  selects  a  goal  or  theme  such 
as,  "Reduce  Unit  Product  Cost  by 
161  by  Year  End."  The  circle  then 
identifies  those  problems  that 
are  keeping  them  from  ach¬ 
ieving  their  goal.  Using  the 
problem  solving  techniques,  the 
circle  members  investigate  and 
analyze  the  problems  they  selected. 
They  request  assistance  from  sup¬ 
port  organizations,  such  as 
engineering,  when  their  investiga¬ 
tions  and  ideas  for  solution  are 
beyond  the  circle's  scope.  Once 
the  circle  arrives  at  a  viable 
solution,  they  implement  it,  if  it 
is  within  their  authority,  or 


present  their  recommendations  to 
management . 

Results 

The  results  of  Quality  Circle 
efforts  within  Honeywell  have  been 
impressive : 

Case  1  -  An  eleatronias  assembly 
department  with  120  employees 
started  10  circles.  Over  a 
three  year  period,  the  circles 
solved  over  230  operational 
problems  and  reduced  the  product 
costs  by  36%. 

Case  2  -  Two  high-volume ,  hybrid 
microelectronic  production  groups, 
with  Quality  Circles,  were  able 
to  achieve  6%,  better  learning 
curves  than  similar  groups  with¬ 
out  circles.  These  improvements 
resulted  in  a  53%  savings  in 
labor  costs  on  one  program. 

Case  3  -  A  heavy  equipment  machine 
shop,  in  a  unionized  plant, 
organized  its  250  employees  into 
12  circles.  Within  the  first 
year,  the  ideas  implemented  by 
the  circles  resulted  in  a  20%> 
improvement  in  machine  utiliza¬ 
tion. 

Case  4  -  A  finance  department 
started  a  circle  in  its  accounts 
receivable  department .  Within 
the  first  six  months,  the  methods 
improvements  implemented  by  the 
circle  resulted  in  a  l3%  increase 
in  invoices  processed  per  day. 

In  case  after  case  we  have  seen 
departments  make  dramatic  improve¬ 
ments  through  the  Quality  Circle 
process . 

The  Avionics  Division  of  Honey¬ 
well  was  the  birthplace  of  the 
company's  Quality  Circle  efforts. 
Today,  the  program  continues  to 
flourish  there  with  over  80  circles 
in  the  manufacturing,  finance, 
engineering,  marketing,  procurement 
and  quality  departments.  In  1979, 
the  direct  labor  and  material  cost 
savings  produced  by  these  circles 
exceeded  $800  thousand  dollars,  a 
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6  to  1  savings  to  cost  ratio. 


At  a  recent  Honeywell  Quality 
Circle  Conference,  Clyde  Molde, 
Vice-President  of  Operations  for 
Honeywell's  Defense  Systems 
Division  attributed  savings  in 
excess  of  $1  million  dollars  to 
the  Quality  Circle  program  in  his 
Division.  ,  He  went  on  to  say  that 
"the  more  significant  outcome  is 
the  way  our  working  style  has 
changed  as  a  result  of  the  pro¬ 
gram  . " 

Bill  Van  Horn,  a  Production 
Manager  in  that  Division,  best 
illustrated  this  change. 

"When  you  sit  down  with  your 
hourly  people  and  really  listen, 
you  realize  that  they  are  as 
smart  as  you  and  have  ideas  as 
good  as  yours.  It's  scary  at 
first,  but  when  you  get  over 
the  fear  and  learn  to  trust 
their  contributions,  you  get 
freed  up  to  do  what  you  were 
hired  to  do  -  manage." 

The  hourly  employees  also  see 
a  number  of  benefits  in  the  effort. 
Bob  Cooper,  a  30  year  Honeywell 
veteran,  who  works  in  the  Avionics 
Division  in  Minneapolis,  compared 
the  traditional  and  quality  cir¬ 
cle  approaches  to  management. 

"Before  the  foreman  passed  out 
what  we  were  supposed  to  do 
and  that  was  that .  Now  we  meet 
in  our  circle  and  decide  what 
we're  going  to  do,  what's  the 
best  way  to  do  it  and  how  we're 
going  to  do  it.  We're  working 
together  with  our  supervisor 
to  make  our  jobs  better  and 
the  company  more  productive." 
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QUALITY  CIRCLES 


Prepared  by:  R.  Barra,  Manager 

Corporate  Product  Integrity 
Westinghouse  Electric  Corp. 

Presented  by:  E.  Mitchell,  Manager 

P.R.I.D.E.  Productivity  Prog. 
Marine  Division 
Westinghouse  Electric  Corp. 

Quality  Circle/  noun  A  small  group  of 
workers,  usually  between  four  and  ten 
people,  who  meet  voluntarily,  on  company 
time,  to  identify  the  causes  of  on-the-job 
problems,  and  to  propose  solutions. 


four  other  people.  The  time  savings:  two 
man-hours  per  shift.  A  Third  Circle  recom¬ 
mended  a  new  method  of  applying  adhesive 
to  printed  circuit  boards  that  cut  defects 
two  hundred  fold. 

The  idea  behind  the  ideas 

Quality  Circles  are  based  on  the  premise 
that  the  people  who  do  a  job  everyday  know 
more  about  it  than  anyone  else.  They  are 
in  the  best  position  to  think  through  a 
problem  and  come  up  with  a  workable  solution 
. . .  particularly  when  productivity  is  in¬ 
volved  . 


You've  probably  seen  definitions  like  this 
one  before.  Simple  ...  succinct  ...  and 
incomplete.  A  Quality  Circle  is  much  more 
than  a  meeting  of  thoughtful  workers.  Look 
inside,  and  you'll  find  a  carefully  structured 
forum  that  encourages  non-management  employes 
to  participate  in  the  decisions  that  affect 
their  jobs.  Look  a  bit  deeper,  and  you'll 
discover  a  powerful  motivational  technique 
that  harnesses  the  creativity,  intelligence, 
and  experience  of  every  Circle  participant. 

Today,  there  are  about  120  Quality  Circles 
throughout  Westinghouse,  including  50  in  the 
Defense  Group.  By  the  end  of  1980,  there  will 
be  a  few  hundred.  And  by  the  end  of  the  dec¬ 
ade,  several  thousand.  We've  made  our  Circle 
program  the  major  "people-oriented"  component 
of  our  corporate  productivity  improvement 
program  for  five  good  reasons:  Quality 
Circles  can  enrich  the  jobs  of  our  employes, 
boost  product  quality,  reduce  manufacturing 
costs,  raise  productivity,  and  ultimately 
improve  our  operating  profits. 


Circular  thinking 


The  traditional  "top-down"  school  of  manage¬ 
ment  holds  that  problem-solving  is  a  manage¬ 
ment  function.  In  other  words:  workers  should 
work  and  managers  should  think.  By  contrast, 
the  "bottom-up"  approach  of  Quality  Circles 
spreads  problem-solving  throughout  an  organi¬ 
zation.  Management  ultimately  makes  the  deci¬ 
sions,  but  employes  take  part  in  the  search 
for  ways  to  improve  their  on-the-job  perform¬ 
ance. 


For  example,  a  Quality  Circle  at  the  Baltimore 
Defense  Electronics  Division  concluded  that 
productivity  would  go  up  if  current  specifi¬ 
cations  and  drawing  requirements  were  summar¬ 
ized  on  a  wall  chart  visible  to  everyone  in 
the  work  area.  Another  Circle  proposed  that 
one  employe  arrive  two-hours  before  a  shift 
change  to  warm  up  processing  equipment  for 


When  employes  are  asked  what  to  do,  instead 
of  being  told  what  to  do,  they  respond  to 
the  challenge  with  more  than  good  ideas. 
Morale  goes  up,  and  with  it  the  team  spirit 
that  motivates  employes  to  be  more  proauctive. 
Employes  take  pride  in  their  work.  They 
develop  a  personal  stake  in  productivity, 
product  quality,  and  reliability.  They 
welcome  advanced  technology  because  they 
feel  "in  charge"  of  it.  There  is  less  re¬ 
sistance  to  change  because  employes  help  to 
initiate  it.  Communications  between  manage¬ 
ment  and  employes  improves  dramatically,  and 
employes  come  to  understand  the  role  they 
play  in  their  organization.  These  intangible 
benefits  will  often  outweigh  the  value  of 
the  ideas  themselves. 

What  a  Circle  is: 

Let's  add  a  few  specifics  to  the  definition 
at  the  top  of  the  previous  page. 

•  The  members  of  a  Quality  Circle  are  co¬ 
workers  in  one  section.  Participation  is 
purely  voluntary;  workers  are  invited  — 
never  compelled  —  to  form  a  Circle. 

Workers  who  choose  not  to  join  are  told 
what  the  group  is  doing,  and  they  are  given 
frequent  opportunities  to  participate  in 
supporting  activities. 

•  A  Circle  usually  meets  one  hour  each 
week,  ideally  away  from  the  workplace  in 
a  meeting  room  set  aside  for  the  purpose. 

•  Every  Circle  has  a  leader  —  usually 
the  section's  first-line  supervisor.  His 
or  her  job  is  to  keep  the  meetings  on 
track,  help  resolve  disputes,  and  guide 
the  Circle  to  appropriate  problems. 

Leaders  arc  trained  to  chair  the  meetings 
in  a  way  that  promotes  the  free  exchange 
of  ideas. 

•  Circle  leaders  go  through  a  preliminary 
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three-day  training  program  taught  by  a 
duality  Circle  facilitator.  This  is  a  man- 
af’e:T.<j;.t- level  employe  who  is  the  formal 
link  between  the  Circles  and  Division 
:canagem**nt.  iie  or  she  assists  the  Circles 
in  th-iL!’  activities;  provides  the  informa- 
'Le.,  materials,  and  other  resources  they 
ne>.  ;;  and  nelps  them  develop  their  presen- 
‘latior.a  to  nanagement.  As  a  rule,  a  facili- 
ta‘  :r  with;  two  or  three  years  of  experience 
;:m  ; t ably  oversee  10  to  20  Circles. 

•  Cnn  facilitator  attends  —  and  helps  to 
—  t;.‘-  -^arly  meetings  of  a  newly 
:  lirci-.  This  gives  the  leader  an 

a.i  .a’l  o.ity  t;  gain  experience. 


ers  are  taught  a  variety  of  funda- 
troi lem-solving  techniques  during 
St  six  or  eight  Circle  meetings, 
.'■.p-e;  data  gathering  and  graphing, 
'jr:',::.;;,  cause-and-effect  analysis, 
etc  analysis.  Ani  they  also  learn 
pr-  sent  their  cor.clusions  and  recom- 


iatier.s  to  management . 


•  A  Circle  will  spei.d  four  to  six  weeks 
analyzing  a  typical  workplace  problem  ... 
aithc..gh  an  experienced  Circle  may  spend 
several  months  solving  a  highly  complex 
problem.  Then  the  facilitator  arranges  a 
meeting  with  the  appropriate  level  of 
management,  so  the  Circle  can  present  its 
proposals. 

"Management  can  accept,  reject,  or  modify 
a  Circle's  recommendations,"  explains 
Ralph  J.  Barra,  Manager  of  Corporate  Product 
Integrity,  and  coordinator  of  our  corporate 
Quality  Circle  program.  "The  Circle  pro¬ 
poses  solutions;  it  does  not  implement  them. 
In  effect,  a  Quality  Circle  multiplies  the 
intellectual  resources  available  within  a 
division  without  diluting  any  of  management's 
authority  or  prerogatives." 

Barra's  group  —  located  at  the  (W)  R&D 
Center  —  has  become  the  in-house  Quality 
Circle  consulting  team,  It  provides  start¬ 
up  advice  and  materials;  trains  new  facili¬ 
tators;  and  keeps  track  of  the  fast-growing 
program  for  the  Corporate  Prod’  ctivity 
Improvement  Committee.  Barra  is  an  enthus¬ 
iastic  advocate  of  Quality  Circles  —  and  a 
storehouse  of  information  for  program 
planners. 

luiding  the  Circle 

".Ttr-jcture  and  direction  are  essential,"  says 
■■ur  ra.  "A  O’jality  Circle  is  not  an  ad  hoc 


management  advisory  committee  or  a  gripe 
session.  The  facilitator  and  leader  are 
the  rudders  that  steer  the  Circle  towards 
job-related  problems  that  its  members  have 
the  expertise  to  solve.  Personnel  and 
industrial  relations  problems  are  off 
limits.  So  are  discussions  about  the  per¬ 
formance  —  or  lack  of  performance  —  of 
specific  individuals." 

A  new  Circle  will  focus  on  simole  workplace 
problems  that  offer  a  high  probability  of 
success.  Early  achievements  build  confi¬ 
dence  and  experience.  Then  the  Circle  can 
tackle  more  ambitious  projects  that  may 
involve  the  cooperation  of  other  sections 
and  other  Circles. 

"In  time,"  says  Barra,  "the  group  will 
shift  gears  from  problem-solving  to 
problem-p> 'eventing.  All  the  while,  the 
members  are  learning  about  their  jobs,  their 
section,  and  their  division.  They  are 
experiencing  personal  growth  and  development. 
And  they  are  learning  to  communicate  with 
each  other,  their  supervisor,  and  their 
management . 

"This  evolutionary  process  generates  the 
long-term  benefits  of  the  Circle  program. 

But  it  can't  happen  unless  senior  division 
management  accepts  three  broad  responsi- 
bilites: 

"First,  management  must  welcome  employe 
participation.  This  means  listening  to  the 
ideas  offered  by  Quality  Circles  . . .  imple¬ 
menting  as  many  of  the  good  ones  as  possi¬ 
ble  . . .  and  explaining  why  rejected  pro¬ 
posals  can't  be  implemented. 

"Second,  management  must  provide  the  infra¬ 
structure  to  support  the  Circle  program. 
Capable  facilitators,  meeting  rooms,  and 
training  materials  are  only  the  starting 
point.  Management  must  encourage  organi¬ 
zations  throughout  the  division  to  cooperate 
with  every  Quality  Circle.  No  Circle 
operates  in  a  vacuum.  Solving  a  problem 
in  Section  A  will  often  impact  the  acti¬ 
vities  of  Section  Z.  Practically  speaking, 
this  demands  the  commitment  of  the  plant 
manager.  Without  it,  the  program  won't 
last  long. 

"Third,  management  must  create  a  system  for 
recognizing  the  achievements  of  its  Quality 
Circles.  Self-respect  and  self-esteem  are 
the  engines  that  power  the  program.  Recog¬ 
nition  is  the  fuel.  There  are  many  ways 
to  show  support  and  applaud  performance. 
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For  example,  senior-management  luncheons 
v;ith  Circles;  mementos  such  as  "T"  shirts 
ar;d  wall  plaques;  articles  in  plant  news¬ 
papers;  yearly  award  banquets;  and  — 
taking  an  idea  from  Japan  —  trips  to  see 
Quality  Circles  at  work  in  other  companies 
at'.d  other  countries." 

A  fevj  fir.ancial  details 

It's  impossible  to  calculate  the  return  on 
investment  for  a  Quality  Circle  program 
because  there  are  so  many  intangible  payoffs 
that  don't  carry  a  price  tag.  Some  ROI 
estimates  at  other  companies  (based  solely 
on  cost  savings  yielded  by  improved  pro¬ 
ductivity)  exceed  500  percent.  However, 
total  total  costs  are  low.  The  major  costs 
of  a  start-up  program  are:  The  facilitator's 
salary  and  benefits;  employe  expense 
associated  with  members  meeting  one  hour  a 
week  on  company  time;  and  travel  expense  and 
training  fee  for  the  facilitator.  A  starter 
kit  of  training  materials  for  five  circles 
is  provided  free,  thanks  to  funding  from  the 
Corporate  Productivity  Improvement  Committee. 

VJhite-collar  Circles 

"The  next  step  in  our  program,"  says  Barra, 
"is  to  expand  the  Quality  Circle  concept 
to  professional  and  support  employes.  White- 
collar  Circles  have  a  less  formal  structure. 
They  can  meet  less  often  than  once  a  week, 
each  meeting  can  last  longer  than  an  hour, 
and  members  may  work  in  different  sections 
...  although  they  will  have  similar  job 
responsibilities.  Problem-solving  tends 
to  focus  on  challenges  that  are  generic  to 
a  profession  or  support  function.  For 
example,  speeding  the  production  of  engin¬ 
eering  drawings,  or  reducing  the  time  to 
prepare  typed  letters.  The  emphasis  is  on 
providing  the  tools  professionals  and 
support  personnel  need  to  do  their  jobs 
better." 

Quality  Circles 

The  first  Quality  Circles  (see  Made  In 
Japan  on  page  16)  worked  to  improve  product 
quality.  "But  we  think  oru  Circle  program," 
says  Barra,  "will  do  much  more.  We  want  to 
improve  the  quality  of  our  processes,  the 
quality  of  our  systems,  the  quality  of  our 
information  resources,  the  quality  of  our 
err^ployes,  the  quality  of  our  management, 
and  ultimately,  the  quality  of  our  company." 


Made  In  Japan 

For  half  of  the  20th  Centrury,  a  "Made  in 
Japan"  label  was  a  warning  to  consumers. 

And  then,  Japanese  industry  invented  the 
Quality  Circle. 

The  story  begins  back  in  the  '50s,  when 
two  American  consultants  brought  Western 
quality-control  techniques  to  Japan.  They 
taught  the  need  for  company-wide  programs 
to  raise  quality,  and  they  stressed  that 
an  entire  work  force  must  be  involved  . . . 
not  just  a  few  designated  quality-control 
managers . 

The  Union  of  Japanese  Scientists  and 
Engineers  agreed.  They  prepared  an  easy- 
to-understand  handbook  on  quality  for 
production  supervisors.  But  few  people 
read  the  text  on  their  own.  And  so  the 
Union  came  up  with  the  idea  of  "reading 
circles";  groups  of  supervisors  who  met 
to  discuss  the  book  and  talk  about  quality. 
Supervisors  liked  the  idea;  they  formed 
reading  circles  with  their  ov/n  workers. 

The  meetings  grew  into  problem-solving 
sessions,  and  the  Quality  Circle  was  born. 

By  one  estimate,  there  are  a  million 
Quality  Circles  in  Japan  today,  involving 
about  ten  million  workers.  But  product 
quality  is  only  one  of  their  concerns. 

Japan  imports  most  of  its  raw  material., 
and  energy;  reducing  waste  and  conserving 
energy  have  long  been  national  priorities. 
Quality  Circles  attacked  these  problems 
in  the  late  '60s.  Many  of  the  solutions 
they  proposed  increased  productivity.  By 
the  mid  '70s,  the  tail  wagged  the  dog: 
Millions  of  Japanese  workers  were  thinking 
together  to  give  Japan  the  fastest-growing 
industrial  productivity  in  the  world. 

Tom  Murrin,  President  of  Public  Systems 
Company (W)  and  Chairman  of  our  Corporate 
Productivity  Improvement  Committee,  sums 
up  the  potential  of  Quality  Circles  with 
this  anecdote:  "A  few  months  ago,  I  asked 
a  leading  Japanese  expert  on  productivity 
a  simple  question;  If  Westinghouse  could 
implement  only  one  program  to  improve  pro¬ 
ductivity,  what  would  you  recommend?  He 
answered  without  hesitation:  'Put  in  a 
Quality  Circles  program.'" 
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BY 

ROBERT  LOZANO,  MA 

A  worrisome  trend  has  recently  be¬ 
come  apparent  in  the  industrial  posture 
of  the  United  States.  Our  productivity 
has  taken  an  uncomfortable  decline  from 
its  annual  average  growth  rate  of  just 
over  3'/.  through  the  sixties  (Time, 

August  27,  1979:  36)  to  .47.  in  T970 
(Ibid.).  This  performance  has  given 
the  United  States  the  lowest  produc¬ 
tivity  growth  of  all  major  industrial 
nations  (Wall  Street  Journal,  21  Feb. 

1 980 ) .  ConFe'rn~has'  ¥1  sb~been  ra i  sed 
over  our  decline  as  a  leader  in  the 
manufacture  of  quality  noods  (Bhote 
1979;  795). 

These  declines  in  productivity 
and  quality  appear  to  be  inextricably 
related.  Poor  quality  products  have  to 
be  remade;  hence  productivity  is 
correspondingly  reduced.  A  third  impor¬ 
tant  component  that  warrents  our 
attention  is  safety,  since  it  has  an 
overlaopinq  influence  on  both  quality 
and  productivity.  It  has  been  our 
experience  that  any  motivation  efforts 
are  easily  erroded  and  neutralized  when 
the  shop  operator  feels  that  he  or  she 
is  working  in  an  unsafe  environment.  No 
number  of  awards  can  alleviate  the 
anxiety  of  a  flash  fire  or  reduce  the 
discomfort  of  an  irritating  chemical 
agent.  The  effect  of  a  safety  incident 
is  further  compounded  when  one  considers 
the  down-time  that  might  be  experienced 
due  to  a  damaged  facility,  personal 
injuries,  unsal vageable  products  and 
tie-up  of  the  appropriate  support 
personnel . 

The  problems  of  maintaining  a  good 
motivation  effort  that  results  in 
quality  cost-effective  products  come  in¬ 
to  sharp  focus  when  we  deal  wi th  man¬ 
rated  space  flight  hardware.  Quality 
and  safety  become  paramount.  We  would 
expect  that  having  the  unique  opportunity 
of  working  with  this  type  of  hardware 
would  provide  the  incentive  to  maintain 


a  high  degree  of  personal  motivation. 
After  all,  how  many  opportunities  are 
availal)le  where  a  person  can  actually 
have  a  hand  in  shaping  the  future? 

We  cannot  categorically  dismiss  the 
absence  of  such  motivation.  It  does 
exist,  but  usually  in  areas  that  have 
experienced  a  history  of  aviation  and 
aerospace  development,  such  as  the  Los 
Angeles  Basin.  At  our  aerospace 
operations  at  the  Michoud  Assembly 
Facility  in  New  Orleans,  we  could 
initially  draw  from  a  pool  of  former 
Boeing  and  Chrysler  employees  experi¬ 
enced  from  the  Apollo  program.  This 
source  of  experienced  personnel  was 
soon  exhausted  and  we  had  to  draw  from 
the  available  labor  market. 

The  New  Orleans  area,  and  indeed 
southern  Louisiana,  has  experienced  a 
long  history  of  agrarian  and  agricul¬ 
tural  products  processing  activities. 

The  area's  two  large  industrial  bases, 
i.e.,  shipbuilding  and  the  petroleum/ 
petrochemical  industries,  have  only 
been  developed  since  1951.  From  these 
two  sources  we  have  drawn  approximately 
357  of  our  floor  personnel.  The 
remaining  657  of  the  work  force  has 
had  very  little  industrial  experience, 
even  less  aerospace  related  experience 
or  no  experience  at  all.  It  is  against 
this  background  of  circumstances  that 
we  frame  our  motivation  effort. 

Our  effort  at  Martin  Marietta 
Michoud  is  founded  on  a  four- faceted 
approach.  A  Manned  Flight  Awareness 
program,  a  Supervisor  Development 
Training  program,  a  Perfonnahee  Award 
plan  and  an  employee  participation 
undertaking. 

Manned  FI i ght  Awareness 

Because  we  are  primarily  an  assembly 
facility,  a  very  large  percentage  of 
our  products  are  manufactured  "out  of 
house."  These  products  range  from 
basic  fastener  hardware  to  primary  sub- 
assembly  components.  This  situation 
has  prompted  the  necessity  of  an 
effective  and  vigorous  supplier- 
oriented  £nid  in-house  Manned  Flight 
Awareness  program. 

0  Supplier  Visits 

Our  current  supplier-directed  effort 
consists  of  providing  a  Manned  Flight 
Awareness  presentation  to  thirty-eight 
major  External  Tank  (ET)  suppliers 
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located  throughout  the  United  States. 

The  presentation  consists  of  a  compre¬ 
hensive  slide  program  that  details  the 
major  Space  Shuttle  components,  i.e., 
the  Orbitor,  the  Solid  Rocket  Boosters 
(SRB's)  and  of  course  our  product,  the 
External  Tank.  This  is  followed  by  a 
profile  of  a  typical  mission  from 
launch  to  earth  orbit. 

During  the  viewing  of  the  mission 
profile  we  highlight  that  moment  during 
the  mission  when  the  supplier's  product 
is  performing  its  function.  We  under¬ 
line  the  criticalness  of  that  particular 
component,  and  point  out  how  its 
performance  relates  to  the  success  of 
the  mission.  At  this  time  a  vast 
majority  of  the  supplier's  personnel 
realize,  for  the  first  time,  what  their 
product  does  and  see  the  results  of 
their  efforts.  Furthermore,  these 
same  people  realize  for  the  first  time 
what  can  happen  if  their  product  fails. 
After  the  slide  presentation  we  display 
a  1/lOOth  scale  model  of  the  Space 
Shuttle  and  conduct  a  question-and- 
answer  session.  This  is  followed  by 
a  motion  picture  of  the  Approach  and 
Landing  Tests. 

We  conclude  our  presentation  by 
giving  each  employee  working  on  the 
product  a  pack  containing  literature  and 
information  on  the  Shuttle,  an  External 
Tank  decal,  a  'first  flight  patch' 
decal  and  a  letter  from  our  vice- 
president  and  general  manager  empha¬ 
sizing  the  importance  of  everyone's 
efforts  to  insure  mission  success. 

0  Supplier  Posters 


have  had  numerous  requests  for  additional 
presentations,  and  we  are  incorporating 
them  in  our  schedule  as  time  permits. 

In  case  after  case  we  fulfilled  a  need 
for  basic  information  and  public 
awareness  of  the  Shuttle  program,  in 
addition  to  raising  the  level  of 
product-  and  job-related  manned 
flight  awareness. 

It  gives  us  a  great  sense  of 
satisfaction  to  know  that  people 
working  on  our  products  now  realize 
that  the  Shuttle  will  be  launched 
from  Pad  39A  at  KSC  instead  of  "off  the 
big  jet."  Perhaps  this  is  indicative 
of  an  information  gap  that  needs  to 
be  bridged,  if  we  seek  public  support 
for  our  space  programs. 

0  In-House  Programs 

Our  product  education  efforts 
at  Michoud  stem  mainly  from  our 
response  to  accommodate  a  large  labor 
force  with  little  or  no  experience 
in  the  aerospace  sector  of  industry. 

This  condition  is  expected  to  remain 
the  same  over  an  extended  period  (Times 
Picayune,  13  April  1980).  Because 
of  this  circumstance,  we  have  to 
direct  our  efforts  to  our  employees 
as  soon  as  they  walk  through  the  door. 

Our  present  posture  includes  a  New 
Employee  Manned  Flight  Awareness 
Orientation,  Manned  Flight  Awareness 
weekly  lunchtime  movies,  ET  roll-outs, 
astronaut  visits  and  in  a  more  general 
motivation  vein,  a  yearly  family  day 
picnic. 

Supervisor  Development  Training 


Very  recently  we  have  added  a  new 
feature  to  our  supplier  visits  to 
reinforce  the  awareness  that  we  want 
to  stress  in  our  presentations.  We 
call  this  feature  a  supplier  poster. 

The  poster  layout  is  dominated  by 
individual  photos  of  all  the  Shuttle 
astronauts  across  the  top  and  bottom 
thirds.  In  the  center  is  a  line 
drawing  of  the  External  Tank  with  a 
circular  highlight  designating  where 
the  supplier's  product  is  located  on 
the  tank.  Alongside  this  display  is 
a  close  up  photo  of  the  product  on 
the  ET. 

0  Results 

Response  to  our  supplier  visits  has 
been  very  good,  indeed  excellent.  We 


The  lack  of  skilled  personnel  also 
impacts  our  supervisory  ranks  at  both 
the  first  and  second  level.  Attrition 
factors  such  as  retirement  and  changes 
of  professions  have  obliterated  the 
supervisory  personnel  that  were  present 
in  the  area  at  the  height  of  the  Apollo- 
Saturn  program.  Some  are  left,  but 
they  are  insignificantly  few  in  number. 
This  situation  has  literally  forced  us 
to  build  almost  all  of  our  first  and 
second  level  supervisory  tiers  in  Quality 
and  Manufacturing.  The  available 
supervisor  candidates  are  usually  people 
from  the  hourly  ranks  that  have  had  some 
time  on  the  floor.  For  the  most  part 
this  leaves  us  with  a  lot  of  young  super¬ 
visors  in  their  twenties  supervising 
their  equally  young  peers.  These 


circumstances  have  produced  some  problems 
in  attaining  a  disciplined  operation 
and  in  developing  a  good  ambience  of 
safety  consciousness. 

Initial  responses  to  this  situation 
include  personal  training  sessions 
conducted  by  some  of  our  Quality 
Chiefs.  On  a  once-a-week  basis, 
seminars  were  held  to  cover  the  basic 
aspects  of  inspection  techniques  and 
methodology.  The  necessity  to  provide 
such  basic  training  reflects,  in  part, 
a  need  for  educational  opportunities  in 
the  public  sector  such  as  those  which 
are  available,  for  example,  in  some 
of  the  Community  College  districts 
(Courses,  1979  -  1980:  p.lOO)  of 
California. 

Our  present  efforts  consist  of  a 
comprehensive  thirteen  course  Super¬ 
visor  Development  Seminar  and  weekly 
supervisor  luncheon  meetings.  The 
training  program  is  the  keystone  to  our 
overall  development  endeavor.  It  is 
administered  by  our  Training  Department 
which  processes  approximately  twenty 
supervisors  every  four  weeks.  The 
courses  include  sessions  in  motivation, 
covering  the  theory  and  its  applications; 
and  Systems  Refinement  Teams,  which  is 
an  introduction  to  Quality  Circles. 

The  weekly  supervisor  luncheon 
meetings  are  designed  to  promote  a 
sense  of  involvement  and  interdepart¬ 
mental  fraternity  among  the  supervisors. 
Typically  the  meetings  consist  of  a 
briefing  by  a  department  manager  and 
his  staff,  detailing  that  particular 
department's  function  and  current 
activities.  After  the  briefing, 
everyone  convenes  in  the  Executive  dining 
room  for  an  informal  luncheon. 

These  meetings  have  elevated  morale 
and  imbued  the  supervisors  with  'Team 
spirit'  because  upper  management  has 
involved  them  in  something  more  than 
just  the  routine  expectations.  The 
results  have  matched  our  original 
goal . 

Performance  Awards 

After  our  personnel  are  'on 
board,'  they  become  eligible  for  any 
one  of  the  three  categories  of  job 
performance  and  special  achievement 
awards. 

0  Spot  Awards 

Recognition  is  given,  at  the 


prerogative  of  the  person's  supervisor, 
for  excellence  that  is  deemed  to  be 
outside  the  scope  of  required  work. 
Awards  given  for  this  recognition 
consist  of  Shuttle  plaques,  pen  sets, 
bronze  medallions  (depicting  the 
External  Tank)  and  sports  event  tickets. 

0  Sustained  Performance  Awards 

When  an  employee  demonstrates  an 
outstanding  work  performance  record  for 
a  period  of  at  least  three  months, 
his  effort  is  recognized  by  our 
sustained  performance  award.  The 
award  is  a  sterling  silver  medallion 
depicting  the  External  Tank  and  is 
usual ly  presented  by  someone  at  the 
managerial  or  directorial  level. 

0  Special  Achievement  Awards 

Supervisors  of  both  hourly  and 
salaried  personnel  can  cite  their 
outstanding  people  who  have  a  record 
of  exemplary  performance  by  nominating 
them  for  candidacy  for  Employee  of  the 
Month  (EOM).  Final  selection  is  made 
by  members  of  the  Manned  Flight  Aware¬ 
ness  Committee,  the  directors  and  our 
vice-president  and  general  manager. 

This  award  carries  with  it  a  plaque, 
a  cash  prize,  reserved  parking  and 
eligibility  for  employee  of  the  Year. 

The  same  Manned  Flight  Awareness 
Committee  convenes  to  select  the 
Employee  of  the  Year.  The  candidates 
are  the  past  Employees  of  the  Month. 

This  award  carries  the  same  perquisites 
as  the  EOM,  except  that  the  cash  prize 
is  doubled  and  the  winner  and  his 
family  are  Launch  Honoree  participants. 

Ranking  high  in  prestige  is  the 
Snoopy  Award.  In  conjunction  with 
our  astronaut  visits,  this  award  is 
presented  to  the  people  who  have  been 
chosen  by  the  Manned  Flight  Awareness 
Committee  for  exemplifying  the  standards 
that  reflect  mission  success. 

Employee  Participation 

We  presently  promote  two  principal 
areas  of  activity  to  provide  direct 
participation  in  people  benefiting 
activities  by  any  employee  on  a 
voluntary  basis.  These  activities 
are  designed  to  encourage  participation 
at  a  decision-making  level  and  consist 
of  the  Employee  Suggestion  System  and 
Systems  Refinement  Team.  The  latter  is 
our  own  particular  brand  of  Quality 


Circles. 

0  Suggestion  System 

We  have  recently  completed  a 
thorough  study  and  assesstrent  of  our 
suggestion  system.  The  outcome  of  this 
study  resulted  in  a  major  overhaul  and 
updating  of  the  existing  plan.  Our 
aim  is  to  provide  a  more  smoothly 
functioning  system  with  a  more  eguit- 
able  payback,  which  will  encourage  great¬ 
er  participation.  Current  data  indicate 
that  in  1979  18'X  of  the  employee 
population  participated  in  the  old 
system.  Of  this  total  737.  was  from  the 
Manufacturing  sector.  Data  gathered  in 
1980  will  be  compared  to  this  baseline 
and  will  indicate  the  effectiveness  of 
our  improvements. 

0  Sys tjm  Refinement  Team^  -  Oua 1 i ty 
Circle 

Our  most  intense  employee  participa¬ 
tion  activity  thus  far  has  been  our 
Quality  Circles  (Juran,  1976),  which  we 
refer  to  as  Systems  Refinement  Teams 
or  simply  SR  Teams. 

The  functional  structure  of  some 
of  our  teams  appears  to  be  unique 
among  Quality  Circles  and  is  discussed 
in  detail  in  recent  papers  (Lozano  and 
Thompson:  In  Press),  (Thompson:  1980). 
Basically,  it  consists  of  a  core  - 
support  operator  concept,  in  which  the 
'hands-on'  operators  form  the  core  of 
the  team  and  work  in  conjunction  with 
representatives  (support  operators) 
from  those  departments  that  interface 
with  them  on  a  day-to-day  basis. 

In  this  manner  many  of  the  barriers 
that  inhibit  communication  among  the 
different  departments  are  surmounted. 

It  has  been  our  experience  that  in  many 
problem  cases,  the  primary  casual 
factors  were  these  barriers  (Ibid:  2). 

The  unique  aspect  of  our  team 
structure  is  the  fact  that  we  have 
extended  this  concept  to  overcome  inter¬ 
company  barriers.  This  development 
evolved  after  we  discovered  that  we  were 
receiving  more  than  907  of  the  components 
for  a  major  assembly  from  a  single 
source,  AVCO  Aerostructures  in  Nashville. 
Our  project  manager  suggested  the 
concept  of  forming  a  corresponding  SR 
Team  at  AVCO  in  order  to  attack  existing 
problems  from  both  ends  of  the  line. 

We  transformed  this  suggestion 
into  a  very  successful  reality. 


Approximately  three  weeks  after  the 
teams  held  their  first  joint  meeting  via 
a  teleconference  hookup  they  resolved 
a  number  of  significant  problems  that 
had  existed  for  years.  On  April  18, 

1980,  these  two  teams  made  a  joint 
presentation  at  our  facility,  where 
they  demonstrated  a  savings  of  at 
least  365  manhours  per  assembly. 

Conclusions 

As  with  any  motivational  endeavor, 
it  becomes  desirable  to  produce  some 
type  of  quantifiable  data  that  allow 
us  to  measure  the  degree  of  success  or 
failure  of  our  efforts.  As  our  multi¬ 
faceted  motivational  program  is 
gaining  maturity,  we  are  assessing 
the  results  in  various  ways. 

Initially  results  of  our  Manned 
Flight  Awareness  program  at  the  suppliers 
were  measured  by  the  complimentary 
response  we  received  from  the  people 
in  attendance  and  by  letters  of 
commendation.  At  present  we  are  taking 
a  more  objective  approach  by  tracking 
the  types  and  frequency  of  non- 
conformancies  as  they  occur  in  the 
periods  before  and  after  our  presentat¬ 
ions.  Data  of  this  type  will  be  forth- 
comi ng . 

Our  in-house  MFA  program  is  producinc) 
the  desired  results.  The  first  line 
supervisors  are  achieving  a  sense  of 
importance  and  worth  with  their 
training.  This  is  instilling  self 
confidence  in  them  which  manifests 
itself  in  better  discipline.  Many 
of  these  gains  are  intangible  and 
therefore  di  fficul  t  to  'measure.  ' 

However,  attitudinal  changes  in  some 
people  have  been  significant  enough 
to  be  apparent. 

The  awards  program  has  just  been 
measured  with  a  survey  that  sampled  307 
of  the  employee  population.  The 
response  indicates  that  707  of  the 
employees  feel  that  the  program  is 
adequate  and  accomplishing  its  intent. 
This  majority  also  feels  that  the  number 
of  awards  presented  (in  all  categories) 
is  not  excessive. 

Recasue  we  are  in  the  process  of 
implementing  an  upgraded  sugnestion 
system,  we  have  not  yet  acquired  data 
to  provide  a  comparison  with  the 
previous  system.  Once  these  data  are 
available,  the  proposed  comparison  will 
reveal  the  effectiveness  of  that  effort. 


Results  from  our  SR  Team  activities 
have  been  both  tanqible  and  intanqible. 

We  have  had  three  manaqement  presentat¬ 
ions  from  the  first  qroup  of  ten  teams 
which  have  been  active  for  approximately 
seven  months.  The  solutions  to  the 
problems  that  were  worked  are  expected  to 
save  hundreds  of  man  hours  per  tank  in 
the  manufacturing  sector  and  an  equal 
number  in  the  support  sector. 

At  AVCf)  Aerostructurefe  there  was  an 
837.  drop  in  defects  on  Shuttle-related 
components.  This  represents  a  decrease 
of  hundreds  of  nonconformities  per 
assembly  and  reflects  the  change  in 
attitude  that  developed  after  the  vast 
majority  of  the  workers  on  that  product 
line  participated  in  SR  Team  activities. 

Dn  the  intanqible  side,  we  have 
seen  a  perceptible  change  in  the 
attitude  of  the  SR  Team  members  that 
reflects  a  sense  of  participation  at 
the  decision-makinq  level.  The  teams 
strive  to  achieve  more  productive 
utilization  of  time  and  work  toward 
the  prompt  resolution  of  safety 
problems.  In  their  reserach  of  problems 
they  have  gained  an  appreciation  and 
awareness  of  managerial  duties  that  was 
not  there  before.  Indeed,  we  have 
seen  complete  reversals  in  the  attitudes 
of  acknowledged  'trouble-makers.' 

The  progress  of  SR  Teams  is  one 
area  that  we  are  prepared  to  measure 
in  depth.  In  addition  to  counting  the 
tangible  gains,  we  are  administering 
an  attitude  survey  that  has  been 
specifically  designed  for  Quality 
Circles.  This  survey  follows  a  general 
format  so  that  data  can  be  compared 
among  different  companies.  We  expect 
this  data  to  be  extremely  useful. 

In  general,  we  are  pleased  with  our 
overall  approach  to  Quality  and 
Productivity  motivation.  We  believe 
that  we  have  some  sound  and  dynamic 
programs  that  are  well-accepted  and 
are  showing  some  very  encouraging 
results.  As  our  measurement  effort 
matures,  we  feel  that  the  quantifiable 
data  will  justify  our  present  enthusiasm. 
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Implemented  to  comply  with  the  revised 
goal  of  mission  success  approaching  100 
percent.  These  changes  Included  new 
technical  and  management  systems,  reviews 
and  procedures  along  with  staff  augmen¬ 
tation  for  the  government,  prime  con¬ 
tractor,  associate  contractors  and  sub¬ 
contractors.  The  experience  sharing  or 
communications  network  utilized  to 
accomplish  the  contemporary  Delta  Is 
treated  herein. 


Introduction 


Network  Approach 


Experience  sharing  has  been  used  In  many 
contexts.  The  term  can  be  roughly 
defined  as  the  communicating  of  knowledge, 
skill,  or  practical  wisdom  derived  from 
observation  of  or  participation  In 
events.  Generally,  it  is  referred  to  and 
interpreted  as  "information  exchange." 

In  a  more  general  sense.  It  can  be 
defined  as  a  "communication  network"  or 
merely  "communications."  This  paper 
presents  the  communications  network  or 
the  experience  sharing  network  as 
employed  on  the  Delta  launch  vehicle 
program. 

Background 

The  Delta  launch  vehicle  program  was 
established  as  an  Interim  launch  vehicle 
program  In  1959.  During  the  intervening 
two  decades,  151  missions  were 
launched  with  139  being  successes 
resulting  in  an  overall  success  rate  of 
92  percent.  This  was  achieved  with  an 
everchanging  vehicle  configuration  being 
upgraded  to  accommodate  major  increases 
in  payload  capability  requirements. 
Through  the  first  102  Delta  launches  In 
the  growing  years  of  the  agency,  the  90 
percent  success  record  was  notably 
acceptable  and  was  achieved  at  a  moderate 
cost.  However,  as  the  payloads  became 
less  replaceable  and  more  expensive,  the 
obvious  conclusion  was  reached--str1ve 
for  100  percent  mission  success  for  space 
launch  vehicles.  This  modification  to 
program  direction  was  effected  on  the 
Delta  Project  In  1974.  For  the  past  49 
launches  within  this  new  period,  the 
success  rate  for  Delta  has  been  96  per¬ 
cent,  an  Increase  of  6  percent  with  only 
one  spacecraft  loss.  This  higher  mission 
success  rate  has  been  the  direct  result 
of  major  program  and  organization  changes 


In  revising  the  program  to  accomplish 
the  redefined  100  percent  Mission  Success 
Goal,  a  total  systems  approach  to  the 
overall  management  of  the  program  was 
employed.  The  program  and  organization 
were  modified  to  include  additional 
reviews,  procedures  and  surveillance  to 
the  point  of  redundancy,  where  necessary. 
The  significance  of  the  tenet  concerning 
"good  people  talking  a  lot"  was  recog¬ 
nized  and  a  greatly  increased  Informal 
communications  activity  was  established. 
A  central  element  identified  for  subs¬ 
tantial  augmentation  was  the  role  of  the 
cognizant-engineer.  The  staffs  were  also 
augmented  in  both  the  industry  and 
government  sectors  and  interfaces  between 
government  and  Industry  were  Increased 
to  establish  very  close  working  rela¬ 
tionships.  In  order  to  retain  the 
greatest  flexibility  while  striving  for 
maximum  assurance  of  program  and  launch 
success,  keeping  in  close  touch  became 
the  maxim  for  all  levels.  Oroanizations 
Internal  to  government  and  industry  were 
modified  to  insure  that  all  project 
members  were  motivated  to  participate 
and  be  responsible  from  Vice  President/ 
Senior  Management  down  to  on-line 
operations  personnel.  The  primary  goal 
was  100  percent  mission  success  and 
Included  was  the  adaptation  of  a 
communications  network  both  on  a  formal 
and  Informal  basis. 

Although  cognizant  engineers  had 
previously  had  a  program  role,  their 
responsibility  was  Increased  extensively 
as  were  the  total  numbers.  These 
Individuals  are  assigned  certain  compo¬ 
nents,  systems  or  subsystems,  and  "live" 
with  them  through  all  aspects  of  the 
program  including  design,  procurement, 
production,  checkout,  launch  preparation. 


launch  and  post-launch  activities.  The 
cognizant-engineers  are  identified  and 
assigned  responsibility  in  both  the 
government  and  contractors'  sectors  and 
function  as  a  team  and  complement  one 
another. 

For  the  contemporary  Delta  the  role  of 
the  Delta  Launch  Vehicle  Engineer  (DLVE) 
was  introduced  into  the- organization. 

The  DLVE  is  assigned  a  launch  vehicle 
early  in  the  production  phase  and  follows 
the  vehicle  through  production,  check¬ 
out,  launch  preparation,  launch  and 
post-launch  activities.  He  essentially 
serves  as  a  systems  cognizant-engineer. 
The  duties/responsibilities  of  the 
cognizant-engineer  in  the  complete  cycle 
are  as  follows: 

0  Assures  Flight  Integrity  of  Critical 
Components 

0  Maintains  Complete  Data  File  for 
Each  Component  Including  Trend 
Analysis 

0  Develops  and  Maintains  Chronology 
and  File  of  Rejection  History 
Including  Failure  Analysis  Results 
and  Corrective  Actions 

0  Informs  Supplier  of  Delta  Flight 
Performance 

0  Periodically  Visits  Manufacturing 
and  Test  Areas  ( In-House/Suppl iers) 

0  Reviews  FAB,  Assembly  and  Test  Data 

0  Approves  Drawing/Process  Changes 

0  Conducts  Failure  Investigations, 

Performs  Failure  Analysis,  Provides 
Failure  Documentation  and  Determines 
Change  Actions 

0  Supports  Program  Readiness  Reviews 

0  Provides  Statement  of  Launch 

Readiness  for  Each  Assigned 
Component  by  Serial  Number  Prior 
to  Every  Launch 

Figure  1  depicts  how  various  elements 
focus  toward  launch  success  motivation 
and  specifically  how  the  cognizant- 
engineer  is  the  central  focal  point. 


These  elements  are  not  organizational 
elements  but  rather  functions  which  are 
included  in  all  organizations.  For 
example.  Program  Management  is  made  up 
of  the  project  manager,  program  office, 
engineering,  quality  assurance,  and 
manufacturing.  All  organizations  have 
a  responsibility,  are  included  in  the 
process  and  are  motivated  toward  the 
single  objective  of  launch  success. 

Network  Elements 

The  major  communications  system  elements 
that  are  utilized  by  the  cognizant- 
engineer  and  the  entire  project  team  can 
be  categorized  into  the  following: 

1.  Change  Management  System 

2.  Quality  Assurance  System 

3.  Nonconformance  Quality  Assurance 
System 

4.  Other  Program  Assurance  Systems 

5.  Informal /Verbal  Communications 

These  elements  are  not  separate  or 
independent  from  each  other  but  are 
highly  interactive. 

1.  Change  Management  System 

The  Change  or  Configuration  Manage¬ 
ment  System  utilized  by  the  prime 
vehicle  contractor  on  the  Delta 
Program  is  illustrated  in  Figure  2. 
This  system  includes  participation 
from  all  organization  and  personnel 
levels  within  the  contractor's 
project  organization.  Additionally 
covered  are  the  interfaces  between 
the  government  and  the  contractor  and 
also  between  the  headquarters  and 
field  organizations  of  the  government. 
The  project  is  managed  by  the  field 
organization  in  concert  with  the 
headquarters  program  office  which 
Interfaces  with  the  vehicle  users  for 
definition  of  requirements. 

The  approval  and  review  process  of  a 
change  order  is  shown  in  Figure  2. 

The  change  authority  can  be  initiated 
by  the  government  or  the  contractor; 
however,  somewhere  in  the  cycle,  the 
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government  must  authorize  it  with  a 
"work  authority."  The  mechanism  for 
the  proposed  change  is  an  Engineering 
Change  Proposal,  (ECP).  Tracing  the 
route  of  the  ECP  illustrates  the 
overall  communications  network 
elements  utilized.  These  include 
reviews  at  all  levels  such  as  Senior 
Review  Board  and  Design  Reviews. 

There  are  different  levels  of  manage¬ 
ment  review/approval  depending  on  the 
type  of  change  and  the  cost  involved. 
The  vehicle  user  (spacecraft  customer) 
gets  involved  in  the  approval /review 
process  for  mission  peculiar  changes 
which  might  affect  or  influence  his 
hardware. 

Figure  2  deals  with  the  prime 
contractor's  system;  however,  the 
systems  for  the  associate  contractors 
are  similar. 

2.  Quality  Assurance  System 

The  Quality  Assurance  System  employed 
by  the  prime  contractor  is  shown  in 
Figure  3.  The  figure  shows  the 
communication  network  that  is  used 
in  the  program  for  both  hardware  and 
software  requirements.  This  is  the 
normal  or  conformance  assurance 
system  and  depicts  the  documentation 
involved,  the  levels  of  review,  and 
the  involvement  of  engineering, 
quality,  production,  management, 
operations  and  customer  interfaces 
in  the  overall  process.  One  can 
readily  see  the  detail,  level,  and 
redundancy  involved  in  the  system 
and  all  involved  share  in  the 
responsibility. 

As  indicated  before,  there  are 
similar  networks  for  the  associate 
contractors. 

3.  Nonconformance  Quality  Assurance 
System 

The  Nonconformance  System  provides 
the  following  key  features  for 
failure  resolution: 

0  Design  and  Reliability  Participa¬ 
tion  in  Failure  Investigation  and 
Resolution 


0  Parts  Specialists  Perform  Failure 
Analysis 

0  Reliability  and  Program  Management 
Approve  Analysis/Investigation  and 
Corrective  Action 

0  Safety  Evaluations  of  Nonconfor¬ 
mance  Reports 

0  Computerized  Data  Storage/ 
Reporting 

0  Failure  Trend  Evaluation  as  the 
First  Step  in  Failure  Investiga¬ 
tion/Analysis 

0  Reliability  Verification  of 
Corrective  Action 

0  System  is  Closed  Loop 

Figure  4  presents  the  communication 
network  flow  chart  which  is  utilized 
when  there  is  nonconformance  in  a 
system,  subsystem,  or  component. 

This  nonconformity  could  be  for  any 
reason;  i.e.,  out  of  specification, 
operational  problem  and  so  forth. 

The  various  organizations  such  as 
quality,  engineering  and  production 
are  involved.  The  process  is 
initiated  by  an  MRR,  which  is  defined 
as  Material  Record  Review.  The  level 
of  management  review  depends  on  the 
problem  and  can  include  senior 
contractor,  government  and  user 
personnel . 

4.  Other  Program  Assurance  Systems 
This  category  includes: 

a.  Preship  Reviews 

b.  Walk  Arounds 

c.  Special  Readiness  Reviews 

d.  T-3  Day  Flight  Readiness  Reviews 

e.  Classic  Reliability  Functions 

f.  Masttr  Line  Checks 

q.  Post  Launch  Fliijht  Data 
Reviews 
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All  of  these  systems  have  been  part 
of  the  Delta  Project  since  program 
inception.  However,  for  the 
contemporary  Delta  many  of  them  were 
modified  significantly.  For  brevity, 
only  the  more  significant  systems 
will  be  discussed  herein. 

The  Preship  Review  is  new  with  this 
era  replacing  the  T-30  day  flight 
review.  The  review  is  now  conducted 
as  it  is  named  before  the  vehicle  is 
shipped  to  the  field  for  launch 
processing  and  readiness.  The 
decision  to  ship  is  made  at  the 
conclusion  of  the  review.  The  review 
is  conducted  at  the  senior  manage¬ 
ment/specialist  level  of  the  joint 
government/contractor(s)/user  team. 
The  specific  areas  covered  are; 

0  Mission  Description 

0  Spacecraft  Description 

0  Mission  Requirements 

0  Vehicle  Configuration 

0  Mission  Peculiar  Analysis 

0  Flight  Mechanics 

0  Structural  Loads  and  Margins 

0  Battery  Margin  Analysis 

0  Vehicle  Checkout  History 

0  Launch  Site  Status 

0  Subcontractor  Status 

0  Associate  Contractor  Status 

The  T-3  Day  Flight  Readiness  Review 
has  always  been  an  important  part  of 
the  program.  However,  for  the 
contemporary  Delta,  it  was  extended 
and  made  more  exhaustive  with  a 
comprehensive  update  of  all  the 
items  covered  in  the  Preship  Review. 
The  review  team  is  made  up  of  senior 
management/ s pecial i St  personnel  from 
the  various  sectors  and,  in  most 
cases,  includes  a  larger  complement 
of  specialists  than  the  Preship 
Review.  The  decision  at  the 


conclusion  of  the  review  is  the 
"go-no-go"  for  fueling  the  vehicle 
for  launch.  All  open  items  and 
discrepancies  including  open  docu¬ 
mentation  items  must  ^  dispositioned 
and  closed  out  prior  to  launcli 
"go-ahead . "  The  major  items  covered 
at  this  review  are: 

0  Mission  Checkout  Status 

0  Significant  Component  Rejection 
History  Update 

0  Failure  Analysis  Review  Update 
0  Nonreproducible  Anomaly  Update 
0  Anomaly  Report  Update 
0  Alert  Impact  Update 
0  Launch  Checkout  Status 

0  Field  Station  Checkout  History 
0  Field  Station  Problem  Review 
0  Events  to  Liftoff 
0  Range  Status 

0  Data  Acquisition 
0  Range  Safety 
0  Weather 

0  Subcontractor  Status 

0  Associate  Contractor  Status 

Another  important  aspect  of  the 
overall  communications  network  or 
experience  sharing  are  the  activi ties 
covered  during  the  Post-Launch  FI ight 
Data  Review: 

0  Technologies  Review  All  Available 
Flight  Data 

0  Flight  Data  Critique  Meeting 

0  Technologies  Summarize  Results 
of  Flight  Data  Review 

0  Identify  Anomalous  Vehicle 
Systems  Performance  in  Final 


Countdown  Minutes  or  Actual 
Flight 


0  Establish  Anomaly  Report 
Assignments  as  Required 

0  Present  Summary  of  Flight  Data 
Critique  Meeting  in  Memorandum 
Form  Management 

0  Close-Out  Anomaly  Reports 

Commensurate  with  Future  Mission 
Applicability 

5.  Informal/Verbal  Communications 

With  the  augmentation  of  the  staffs 
for  the  government  and  industry 
teams,  the  joint  management  team 
issued  direction  to  personnel 
encouraging  more  informal  or  verbal 
communications.  For  the  contemporary 
Delta  to  achieve  its  mission  success 
goal  of  100%,  it  was  recognized  that 
peer  groups  and  counterparts  had  to 
be  in  constant  communications,  via 
telecon,  meetings  and  the  like. 

Since  the  prime  government  and 
industry  teams  are  a  continent  apart, 
this  is  of  utmost  importance. 

The  results  have  proved  most 
favorable.  The  amount  of  time  spent 
on  the  telephone  has  increased 
significantly  and  so  has  the  time 
devoted  to  informal  meetings  and 
reviews.  Personal  relationships 
between  government  and  industry 
counterparts  have  matured  into 
mutual  respect,  trust  and  excellent 
rapport.  The  intangible,  positive 
results  are  noteworthy.  Certainly, 

I  these  informal  communications  have 

not  replaced  the  formal  correspon¬ 
dence  requirements  which  are 
necessary  and  important;  but,  the 
program  has  benefited  considerably. 

Conclusions 

I  - 

The  network  as  described  and  applied  to 
the  Delta  Launch  Vehicle  Program  for 
communicating  and  sharing  work  experience 
has  had  positive  measurable  results. 

The  establishment  of  the  Delta  Launch 
I  Engineer  and  the  amplified  role  of  the 

cognizant-engineer  were  the  two  major 


mechanisms  used  to  enhance  the  communica 
tion  network  of  the  Delta  Program.  This 
enhancement  coupled  with  indepth  reviews 
formal  and  informal  communications, 
and  efficient  organization  changes  have 
played  a  significant  role  in  increasing 
the  Delta  launch  success  rate  from  90 
percent  to  96  percent. 
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SUMMARY  OF  ISSUES 
AND  RECOMMENDATIONS 


MOTIVATION,  TRAINING  AND 
EXPERIENCE  SHARING 

SUMMARY  OF 

ISSUES  AND  RECOMMENDATIONS 


1.  Top  Management  Commitment 
Issue 

Top  management  in  most  organi¬ 
zations  lack  a  firm  commitment 
to  quality. 

Recommendations 

o Government  and  industry  assoc¬ 
iations  should  emphasize  manage¬ 
ment  responsibility  to  focus  on 
product  quality  as  a  major  objec¬ 
tive. 

2.  Experience  Sharing 
Issue 

There  is  a  deficiency  in  documen¬ 
tation  and  broad  dissemination  of 
information  on  success/failure 
experience  and  related  subjects 
throughout  the  Government/Indus - 
try  team. 

Recommendations 

o  Establish  a  small  government/ in¬ 
dustry  committee  to  study  this 
problem  and  develop  short/long- 
range  solutions. 

3.  Training  -  Motivation 
Issue 


training  programs  in  their  pro¬ 
posals. 

o Government  should  develop 
recognition  programs  for  con¬ 
tractors  who  demonstrate  effec¬ 
tive  and  innovative  training  and 
motivation  programs. 

o  Government  should  publicize 
those  programs  which  it  finds 
have  demonstrated  superior  train¬ 
ing  and  motivational  effective¬ 
ness. 

4.  Funding  Motivation 
Issue 

Motivation/Problem  -  solving 
activities  are  often  not  contract- 
urally  funded. 

Recommendations 

oSuch  efforts  should  be  supported 
by  DOD. 

5.  Sub  Tier  Suppliers 
Issue 

Sub  tier  and  bulk  suppliers  are  a 
major  factor  in  high  defect  rates, 
due  to  insufficient  motivation  to¬ 
ward  achieving  high  quality  in 
products. 

Recommendations 

oinstitute  a  joint  government/  in¬ 
dustry  campaign  to  emphasize 
the  importance  of  quality. 


Greater  emphasis  on  personnel 
training  and  motivation  is  needed 
to  insure  improved  productivity 
and  quality  in  space  system  pro¬ 
grams. 


Recommendations 

o  Prospective  contractor  should  be 
asked  to  submit  their  plans  for 


CLOSING  REMARKS 


Maj.  Gan.  Janvaa  W.  Stanibarry 
OCS/Contracting  and  Manufacturing 
Air  Forca  Syitamt  Command 


CLOSING  ADDRESS 

MaJ.  Gen.  James  U.  Stansberry 
DCS/Contract1ng  and  Manufacturing 
Air  Force  Systems  Command 

Our  nation  has  a  very  serious  problem. 
Things  cost  too  much  and  they  don't 
always  work  right.  That  Is  described 
by  people  who  make  more  than  40  cents  an 
hour  as  a  big  productivity  problem. 

Often  times  In  Mission  Assurance  or 
Quality  Assurance  we  get  very  narrow  In 
our  definition. . .maybe  too  narrow.  I 
was  delighted  to  see  that  this  conference 
talked  about  Mission  Assurance  as  It  did 
last  time.  I  will  submit  to  you  that 
there  can't  be  Important  productivity 
without  Mission  Assurance,  meaning: 
does  the  stuff  work  as  IntendedI  If 
you  think  In  terms  of  productivity  (you've 
already  looked  at  the  charts,  I'm  told, 
earlier  In  the  week)  you'll  see  that 
the  United  States  Is  not  doing  such  a 
red  hot  Job  In  terms  of  productivity. 

In  the  decades  of  the  60s  the  annual 
average  growth  rate  of  productivity  per 
worker,  as  measured  by  output-over* Input, 
was  about  3%  on  a  national  scale.  If 
you  look  In  the  decades  of  the  70s,  It 
was  about  2*.  You  know  the  tendency 
that  I  would  have,  and  I  think  you  would 
have.  Is  to  say,  "Well,  a  \%  drop,  what 
the  hell  It's  only  IX".  But  If  you  equate 
that  to  IX  of  what,  In  terms  of  our  Gross 
National  Product,  our  output.  It  means 
that  somewhere  along  the  way  In  the  last 
ten  years,  when  all  of  us  were  working 
so  hard,  we  lost  $400  billion.  Those 
dollars  are  enough  to  do  a  lot  of  the 
things  that  we  with  our  orientation,  AIA, 
NSIA,  NASA,  and  DOO,  think  very  Important 
for  this  country  to  do.  The  400  billion 
bucks  are  gone  I 

Last  July  I  was  sitting  In  front  of  my 
television  set  having  my  evening  work 
pill,  and  I  heard  President  Carter.  He 
had  Just  come  down  from  Camp  David.  He 
was  talking  about  the  necessity  of 
restoring  the  country's  spirit.  He  said 
that  the  productivity  of  the  American 
worker  Is  actually  going  down.  That's 
got  to  be  a  very  serious  claim  to  those 
of  us  Interested  In  mission  assurance. 

It's  a  national  problem  that  probably 
requires  national  solutions,  and  our 
leaders  have  to  devise  solutions  at  the 
top.  But  there  Is  darn  little  that 
they  can  do  unless  they  receive  a  lot  of 


push  from  the  bottom.  In  looking  over 
the  results  of  your  workshops  last  night 
and  this  morning.  It  struck  me  all  here 
are  pushing,  as  I  am  at  our  level.  That 
push  has  to  result  In  Increased  producti¬ 
vity  If  we  are  going  to  continue  to 
defend  this  country  properly.  I  cannot 
subscribe  In  any  way  to  any  thesis  that 
says  that  you  can  be  a  first  rate  military 
power  -  and  second  rate  In  Industrial 
productivity.  It  Is  not  going  to  happen. 
So  you  start  thinking  a  little  bit  about 
what  we  can  do  about  It.  Productivity, 
according  to  all  the  experts,  and  I'm 
no  one,  -  the  people  over  In  labor  - 
people  In  the  Department  of  Commerce  - 
people  In  government  -  people  In  academia 
all  seem  to  agree  that  productivity  comes 
from  technology.  It  comes  from  capital, 
and  It  comes  from  labor.  And  again.  If  I 
look  over  your  workshops,  I  note  the 
emphasis  that  you've  accorded  to  these 
three  areas,  probably  more  on  technology 
and  labor  than  on  capital,  and  considering 
the  gathering,  I  think  that's  quite  appro¬ 
priate.  So  we  see  a  national  problem  that 
has  a  great  Impact  on  the  Department  of 
Defense  and  on  the  Air  Force.  That  prob¬ 
lem  sets  the  mood,  sets  the  environment  In 
which  we  make  our  decisions. 

Now  lets  move  from  the  national  level. 
Perhaps  we  should  discuss  the  Air  Force 
concerns.  That  makes  me  think  about 
my  boss.  General  Slay,  and  by  the  way, 
let  the  record  reflect  that  I  never 
never,  ever  again  want  to  hear  the  term 
"Slay  Initiative".  Boy,  am  I  up  to  here 
with  thatl  I  say  that  not  because  my 
retirement  papers  are  going  to  be  submit¬ 
ted  at  noon  (hopefully  they  aren't), 
but  because  the  things  we  are  trying  to 
do  In  the  area  of  productivity.  In  the 
area  of  manufacturing.  In  the  area  of 
quality  assurance,  reliability,  main¬ 
tainability  and  our  contractual  methods 
are  not  "Slay  Initiatives".  They  are 
"Air  Force  Initiatives",  the  whole  Air 
Forcel  General  Kelly  Burke  testified 
to  that  effect  In  Congress  a  couple 
of  weeks  ago.  So  If  you're  sweating  out 
Slay  retiring,  or  Stansberry  retiring, 
find  another  solution.  The  Air  Force 
Initiatives  are  going  to  be  with  us  a 
long  time.  I  think  you'll  decide  they 
aren't  so  bad  If  we  get  a  chance  to 
talk  more  to  each  other  about  them. 

From  the  viewpoint  of  the  command  now, 
we  have  to  have  an  obvious  Interest  In 
productivity.  In  component  parts. 


mission  assurance,  quality  -  why? 

Obvious:  We  spend  $16  billion  a  year  on 
hardware;  we  have  300,000  contractual 
actions  a  year.  Of  course  most  of  those 
are  little  ones,  you  know,  beans  and  bul¬ 
lets—  but  we  have  about  2,000  contract¬ 
ual  actions  at  over  $10,000  apiece,  and 
we  have  maybe  50  or  60  that  get  way  up 
there  over  the  $100  million  mark.  A  lot 
of  money  to  be  spent  for  a  lot  of 
hardware. 

Like  other  customers,  our  hardware  doesn't 
always  work  right  either.  General  Henry 
and  others  within  government  and  industry, 
have  shared  with  each  other  during  the 
last  few  days  some  of  the  problems. 

These  problems  are  not  Isolated  to 
strictly  space  vehicles,  satellites,  and 
missiles;  we've  had  problems  that  fall 
across  the  board. 

Sometimes  the  problems  are  the  little 
things  that  shouldn't  go  wrong.  Why 
should  aluminum  be  soft?  You  wouldn't 
think  that  after  all  the  years  of  proces¬ 
sing  aluminum  we  would  get  a  big  batch 
into  our  system  that  hadn't  been  properly 
quenched,  and  as  a  result  lost  some  of 
its  hardness.  A  real  problem  for  the 
Air  Force  and  other  services.  We've  had 
airplanes  whose  controls  Jammed  because 
a  sealed  area  was  sealed  with  dirt  trapped 
inside.  It  hadn't  been  cleaned.  We've 
had  engines  on  airplanes  take  off  without 
the  airplane.  I  don't  wish  to  go  on  with 
any  litany  of  troubles. 

The  other  side  of  the  coin  has  to  be  that 
American  arms  are  the  standard  of  the 
world.  But  we  still  have  a  long  way  to 
go.  From  the  perspective  of  the  command, 
we've  got  a  big  chunk  of  the  act  -  16 
billion  bucks  worth  of  hardware...  We  want 
to  spend  it  wisely,  we  want  what  we  paid 
for  to  work  properly  -  a  very  simple, 
straight  forward  approach.  Now  you  might 
say  at  this  point  (I  said  I'd  tell  vou 
a  little  about  personal  perspective), 
why  is  Stansberry  talking  about  all  this 
quality  stuff?  He's  never  been  in  Quality 
in  his  whole  lifel  But  I  want  to  tell  you 
I  have.  My  relation  to  Quality  Assurance, 
and  this  point  is  key  to  my  pitch,  started 
when  I  was  about  14  years  old.  My  mother 
worked  in  a  QA  work  force  in  a  plant  in 
Buffalo  that  produced  depth  charges. 
Naturally,  as  a  young  teenager  in  World 
War  II,  I  was  pretty  proud  of  the  old 


lady  working  at  making  sure  those  things 
worked  right.  Even  they  had  problems. 

It  must  have  been  the  height  of  aggrava¬ 
tion  to  go  find  an  eneiny  submarine  and 
shoot  a  lot  of  depth  charges  at  them  and 
find  the  depth  charges  didn't  work  I  What 
was  the  problem  there?  The  gauges  that 
were  used  to  clear  parts  were  not  properly 
calibrated.  A  little  thing.  A  lot  of 
people  out  there  staking  their  lives  on 
a  mission,  and  because  of  a  failure 
that  hardware  did  not  work.  Later  I  had 
an  opportunity  to  serve  in  a  Japanese 
aircraft  factory  -  Kawasaki  -  and  I  came 
away  from  there  thinking  "No  wonder  the 
Japanese  proved  to  be  such  a  potent 
fighting  force  in  World  War  11".  Those 
people  understood  quality  workmanship. 

We  were  building  T-33s  on  a  government- 
to-government  arrangement  and  their  T-33s 
were  a  little  faster  than  Lockheed's  and 
the  reason  was  the  care,  the  extra  time 
that  went  into  the  workmanship  that  went 
into  the  product.  I  came  home  (20  years 
ago)  thinking  it's  not  going  to  be  long 
before  the  whole  world  realizes  that  the 
Japanese  understand  quality  workmanship. 

Those  figures  we  talked  about  in  terms 
of  productivity  -  I  was  sort  of  shocked 
to  hear  the  other  day  where  the  Americans 
are  Increasing  their  productivity  1.8 
to  2%  a  year,  the  Japanese  are  increasing 
theirs  10$  a  year.  And  last  year  we  had 
to  send  a  team  over  to  Japan  and  ask 
them,  "Please  explain  to  us  again  how 
to  produce  quality  goods."  That  was  kind 
of  interesting.  We  know  how,  but  some 
how  or  another  we've  forgotten.  And  now 
we  are  rediscovering  what  it's  really 
like.  Not  only  are  we  rediscovering 
in  our  own  little  circles  but  I  think 
it's  very  heartening  that  now  nationally 
we  are  seeing  the  words  productivity  and 
quality  assurance  pop  up.  Yesterday 
on  the  airplane  I  was  reading  Time  maga¬ 
zine,  I  Just  had  to  bring  this  along,  and 
I  suggest  maybe  you  ought  to  get  a  copy. 
"Stunning  Turnaround  in  Tarrytown"  showing 
how  General  Motors,  at  a  plant  in 
Tarrytown  had  a  lot  of  problems.  Workers 
and  bosses  were  constantly  at  each  other's 
throats;  employee  grievances  outstanding 
against  management  totaling  2,000; 
tremendous  reject  rates;  and  Inside  a 
couple  of  years  they  turned  it  around  1 
Part  of  this  answer  -  it  doesn't  say 
this  in  the  article  -  is  the  quality 
circles  proposition.  They  got  the  workers 


and  the  first  line  supervisors  and 
gave  them  a  chance  to  talk  to  each 
other  about  the  job  during  company  time. 
And  somehow  or  another  the  thing  started 
to  happen.  For  example,  they  went  into 
one  unit  that  had  30  windshield  installers. 
After  the  workers  had  been  disciplined 
during  the  previous  six  months  for  poor 
work,  during  discussion  it  was  revealed 
that  each  worker  selected  a  different 
point  around  the  windshield  to  begin 
applying  the  seal.  One  worker  explained 
that  he  started  where  the  radio  antenna 
wires  emerged  from  the  windshield  because, 
"To  get  a  little  extra  adhesive,  a  puddle, 
and  that  stops  leaks."  Everybody  stood 
around  amazed,  saying  how  simple  and 
so  right,  and  that's  where  the  leaks 
are!  They  all  decided  that  they  would 
all  do  it  that  way  and  the  reject  rate 
went  down  dramatically,  almost  instantly. 

That  wasn't  such  a  big  deal,  but  what  is 
a  big  deal  that  Time  magazine  and  the 
national  publications  are  beginning  to 
react  to  the  problems  of  American  produc¬ 
tivity,  American  quality.  I  think  we  are 
going  to  see  a  tremendous  turnaround  in 
the  next  several  years.  I  think  we'll 
soon  get  to  the  point  (it  might  take  us 
several  years)  that  once  again,  American 
products  are  the  standard  of  the  world. 
American  productivity  is  what  everyone 
shoots  for.  American  quality  is  number 
one!  It's  really  got  to  happen. 

In  any  event,  with  some  of  this  interest 
in  quality,  I  volunteered  for  a  job 
once,  chairman  of  the  DOD  Quality  Assur¬ 
ance  Council!  I  lasted  one  day  and  was 
transferred.  The  heart  of  the  problem 
that  the  quality  assurance  work  force  has 
always  had  is  summarized  in  this 
question  -  who  is  Mr.  Quality  Assurance 
in  the  Department  of  Defense?  Ask  that 
question  of  most  people  and  you'll  get 
an  answer  kind  of  like  "Gee,  I  know  who 
he  used  to  be  but  he  got  sick".  Some¬ 
thing  like  that.  We  don't  have,  haven't 
had  anybody  at  the  top  around  whom  all 
people  concerned  with  quality  can  rally. 

We  haven't  had  a  daddy  rabbit;  we  haven't 
had  a  strong  voice... 

(At  this  point.  General  Stansberry  cited 
the  Quality  Horizons  report  written  by 
a  group  headed  by  Col  B.  Weiss.  One  of 
the  recommendations  was  to  appoint  a 
chief  of  Product  Assurance  at  the  command 
level.  General  Stansberry  stated  that 


such  a  position  would  be  created  and 
staffed  before  the  summer  was  out.  He 
went  on  to  describe  the  type  of  indivi¬ 
dual  needed  for  this  job.) 

We  know  that  he's  going  to  have  to  be  a 
special  kind  of  guy.  He's  going  to  have 
to  be  able  to  understand  the  techniques 
he'll  be  working  with  -  he's  going  to 
have  to  be  able  to  handle  the  power 
he'll  be  given.  He'll  have  to  be  a  very 
articulate  and  able  person.  It  will  be 
a  small  office  that  will  encompass 
reliability,  maintainability  and  quality 
assurance  at  least  initially.  It  will 
be  a  small  office  with  a  great  deal  of 
authority  and  power. 

The  second  thing  that  we  are  going  to 
do  is  to  try  to  correct  the  imbalance 
between  the  front  end  and  the  back  end, 
and  put  more  emphasis  on  the  engineering 
on  the  front  end,  and  less  emphasis  on 
the  inspection  of  the  back  end.  I  remem¬ 
ber,  as  a  captain  at  Kawasaki,  I  was  new 
at  all  this  stuff.  I  was  in  there  a 
couple  of  days,  when  I  decided  that  one 
thing  you  are  supposed  to  do  is  go  out 
and  look  at  the  airplane  now  and  then  if 
you  are  supposed  to  be  the  Air  Force 
plant  representative.  I  went  out  there 
a  few  times,  and  I  never  saw  any  of  my 
own  QA  guys  out  there .  I  went  back  to  the 
head  of  our  QA  and  asked,  "Where  are  our 
guys?  I'm  up  here  looking  at  airplanes 
and  I  never  see  any  of  them  looking  at 
airplanes."  He  said  "Hey,  we  inspect 
the  contractor's  system",  (and  think 
about  that  a  little)  and  sure  enough 
they  were,  reading  technical  orders  and 
procedures  and  processes.  That's  a 
pretty  good  answer  but  after  awhile  you 
say  it  sounds  too  good.  So  I  told  them, 
"Inspect  your  system  by  going  out  to 
look  at  the  airplanes  at  least  occasion¬ 
ally".  I  think  some  product  orientation 
basis  will  always  be  needed.  So  don't  do 
away  with  important  back  end;  it's  just  a 
matter  of  balance. 

The  third  thing  we  are  going  to  do  is 
be  a  lot  more  attentive  to  the  needs  of 
our  work  force  in  terms  of  training. 

We've  really  done  a  poor  job,  I  think, 
in  DOD  -  certainly  at  Air  Force  Systems 
Command,  of  offering  training  and  upgrad¬ 
ing  of  our  work  force.  They  didn't  get 
any  training  to  speak  of  in  software,  and 
look  at  all  of  the  software  problems  we 


face.  If  you  look  at  the  profile  of  the 
whole  work  force,  they  are  distinctly, 
definitely,  by  anybody's  definition, 
undertrained.  Ther  have  been  very  few 
people  coming  in  to  our  work  force.  It's 
a  poor  place  to  be.  How  do  you  ever  get 
promoted  if  there  is  nobody  on  top 
taking  care  of  you?  And  we've  had  a  few 
people  coming  in.  This  fall  we'll 
launch  a  quality  intern  program. 

We're  going  to  bring  new  people  into  work 
force  as  interns,  train  them  using  our 
resources.  I'm  sure  tapping  industry  re¬ 
sources  and  using  the  resources  of  other 
government  agencies.  It  will  start  off 
relatively  slow,  but  we  will  still  have  30 
or  40  new  people  in  the  work  force  by  the 
time  late  fall  or  Christmas  comes  around. 
We're  very  excited  about  that. 

We're  going  to  do  a  much  better  job  of 
linking  our  business  methods  with  our 
technical  methods  and  you've  heard  about 
this  in  terms  of  product  assurance  para¬ 
meters,  or  perhaps,  called  a  warranty.  As 
a  result  of  this  recommendation  we  said, 
"Look,  maybe  we  ought  to  get  together 
and  talk  about  this  subject  while  we  are 
not  in  the  heat  of  completion  or  negotia¬ 
tion  We  got  representatives  of  several 
companies  together  and  said,  "What  would 
be  a  good  way  for  the  Air  Force  to  go 
about  getting  more  assurance  from  the 
contractor  that  the  stuff  is  going  to 
work? 

We  made  up  several  important  first 
principles  to  start  with.  One  of  the 
principles  was  good  things  cost  money  at 
the  front  end,  and  the  customer  better 
understand  that.  We  do  understand  it, 
and  we're  willing  to  pay  it.  After  laying 
out  these  first  principles,  we  investi¬ 
gated  32  candidate  programs  where  we 
thought  a  form  of  warranty  or  a  form 
of  incentive  could  be  used.  We  got  a  lot 
of  very  valuable  input  from  industry  and 
we  are  now  pursuing  that.  In  addition, 
we  took  the  essence  of  our  deliberations 
over  several  meetings  and  reduced  it  to 
a  handbook,  our  people  in  the  program 
offices  can  use  and  industry  can  use  in 
negotiating  sound  product  performance 
parameters  for  use  in  our  contracts.  I 
think  this  offers  great  promise.  What 
we  don't  want  is  something  to  break  and 
then  stick  to  the  contract  of  the  60s. 

What  we  do  want  is  for  it  not  to  break 
and  we've  had  some  experience. 


The  ARC/ 164  Radio  comes  to  mind.  A  few 
years  ago  we  had  radios,  UHF  radios  in 
our  aircraft  that  had  a  mean-time-between 
failures  of  30  to  50  hrs.  of  operation. 
This  was  just  eating  us  alive  with  main¬ 
tenance  costs.  After  some  competitive 
bidding  and  prototying,  we  bought  a  radio 
from  Magnavox  and  an  appropriate  incentive 
in  the  contract,  laid  on  them  what  some 
people  thought  was  a  laudable  requirement. 
We  now  have  the  ARC/164  radio  with  a 
mean-time-between-fai lures  of  over  700  hrs 
It  can  work!  It  takes  some  resolve  on 
the  part  of  the  customer,  it  takes  some 
resolve  on  the  part  of  industry.  We're 
going  to  get  there. 

Now  my  over-all  message  to  you  is  not  any 
of  these  little  details.  It  really  goes 
like  this... we  perceive  a  national 
problem  -  the  problem  of  productivity  in 
the  US.  We  are  not  going  to  sit  by  in  the 
Air  Force  and  leave  it  unaddressed.  We 
are  willing  to  devote  our  money  and 
attention  to  technology,  making  capital 
available  (front-end-money)  helping,  when¬ 
ever  we  can,  our  labor  force  to  do  it 
right  -  because  they  want  to  -  I'm  convin¬ 
ced  of  that!  The  days  when  a  guy  can  say 
"Hey,  who  the  hell's  in  charge  of  Quality 
Assurance  around  here?",  and  not  get  an 
answer,  are  over,  as  far  as  the  Air  Force 
Systems  Command  is  concerned. 


CLOSING  REMARKS 

Col.  Norman  Niederman 
Director,  Manufacturing  and  Quality  Assurance 
A.  F.  Space  Division  (AFSC) 


CLOSING  BUSINESS 

(Col.  Nonnan  Neidennan) 

As  a  chairman  of  this  conference  plus  being 
the  anchorman  on  the  Agenda,  I  have  a  doubly 
rewarding  position,  and  as  my  wife  well 
knows  having  the  last  word  in  any  discussion 
is  most  advantageous.  So,  I  plan  to  make 
the  most  of  it! 

General  Henry  said  the  first  day  -  "Finding 
iolutions  to  Mission  Assurance  Problems  is 
our  business".  We  have,  in  just  one  week, 
identified  numerous  problems  that  will  take 
time  to  resolve.  What  is  done  to  implement 
the  recommendations  proposed  by  the  individ¬ 
ual  workshops  is  up  to  each  and  everyone  of 
us.  What  we  do  will  very  well  be  reflected 
on  future  activities  in  space.  How  we  fol¬ 
low  and  embrace  the  guidelines,  recommend¬ 
ations,  procedures  and  standards  that  have 
evolved  from  this  Mission  Assurance  Confer¬ 
ence  will  be  the  ultimate  test  of  how  suc¬ 
cessful  this  conference  has  been. 

What  I'm  saying  is  we  still  have  a  lot  of 
work  to  do.  We  can't  walk  away  from  this 
meeting  and  think  that  the  issues  we've 
discussed  will  take  care  of  themselves,  or 
that  the  problems  will  disappear.  I  dare 
say  that  most  of  us  will  at  one  time  or 
another  be  directly  or  indirectly  involved, 
and  when  you  are  -  step  up  and  accept  the 
challenge.  We  collectively  can  make  the 
difference  between  the  improvement  "f  our 
track  record  in  Space,  or  the  degradation 
of  our  capabilities  and  available  resources. 
Many  of  you  will  be  called  on  to  help  in 
completing  tasks  or  action  items.  When  you 
are,  please  help. 

As  for  your  immediate  activities  (speaking 
for  the  sponsors  of  this  conference)  we 
intend  to  read,  analyze,  and  respond  to 
each  and  every  critique  sheet  as  well  as 
the  workshop  recommendation.  At  the  con¬ 
clusion,  today,  we  will  start  compiling  the 
proceedings.  They  will  be  edited,  and  re¬ 
typed,  if  necessary,  and  published.  All 
conference  registrants  will  receive  a  copy. 

We  are  also  going  to  prepare  an  executive 
summary  in  May.  If  you  would  like  to  get 
a  copy  of  this,  please  place  your  name  and 
address  on  a  separate  piece  of  paper  that 
says,  "Executive  Summary",  located  at  the 
Registration  Desk. 


I  would  like  to  especially  thank  General 
Henry,  Mr.  Plummer,  General  Robinson,  and 
General  Stansberry  for  taking  the  time  from 
their  busy  schedules  to  talk  to  us.  I 
would  like  to  thank  the  opening  day  panel 
on  past  and  future  challenges  of  Mission 
Assurance.  They  were  truly  inspirational. 

I  would  also  like  to  thank  the  chairmen  who 
so  adroitly  conducted  their  workshops  -  and 
the  hardworking  coordinators  who,  up  until 
the  conference,  made  sure  the  chairmen  were 
on  schedule  with  all  of  the  many  tasks  asso¬ 
ciated  with  developing  a  workshop.  And  to 
the  planning  committee  who  worked  so  hard 
and  met  so  many  times  to  assure  this  con¬ 
ference  was  a  success  -  Thanks.  Our  Admin¬ 
istrative  staff  also  deserves  sincere  thanks. 
And  lastly,  the  participants,  the  speakers 
and  the  spoken  to,  without  you  there  would 
not  be  a  conference  -  thank  you. 

I  believe  we  have  a  great  team  effort  ai  .  I 
am  personally  confident  that  this  Air  Force/ 
NASA/Industry  approach  to  solving  our  con¬ 
cerns  will  be  most  rewarding.  I  also  want 
to  recognize  the  National  Security  Industri¬ 
al  Association  and  the  Aerospace  Industries 
Association  for  sponsorint  this  conference, 
and  the  continuing  contribution  that  these 
organizations  are  making  to  the  advancement 
of  national  objectives. 

So  please  let  me  remind  you,  this  is  not  the 
end  of  the  conference  activity,  but  the  con¬ 
tinuing  effort  to  assure  that  through  creat¬ 
ivity,  innovation,  enthusiastic  implement¬ 
ation  both  by  industry  and  government,  and 
perceptive  and  dynamic  management,  we  can 
achieve  Mission  Assurance.  And  now,  I  de¬ 
clare  this  meeting  adjourned. 
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